{"cells":[{"cell_type":"markdown","metadata":{"collapsed":false,"id":"D89B0EAB542642F98900CCC4D4E49360","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"# Lecture 7: Markov Chain Monte Carlo (MCMC) \n## ——现代贝叶斯分析的内核\n\n## Instructor： 胡传鹏（博士）[Dr. Hu Chuan-Peng]\n\n### 南京师范大学心理学院[School of Psychology, Nanjing Normal University]"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"D610CFFFE4A34F338C363D06CDCAE682","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"课前回顾: \n\n- 概率基础;\n\n- 贝叶斯公式中各个部分的意义\n\n- 理解后验分布及其计算"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"2D69EF0742AE4913866F72A6BE6D1B16","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"## $p(\\theta|data) = \\frac{p(data|\\theta)p(\\theta)}{\\pmb {p(data)}}$"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"080B16A5C19948DAAA876A41FC47A2DB","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"使用贝叶斯公式的目标：\n\n## $\\pmb{p(\\theta|data)} = \\frac{p(data|\\theta)p(\\theta)}{p(data)}$ "},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"24C01E78CC9A4A048720E8FB099769E9","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"**以探究地球海水覆盖率为例。**\n\n- 假设我们在获得数据以前的**先验**认为：地球海水覆盖率$\\theta$为0-1的可能是相等的。$\\theta \\sim uniform(0,1)$\n\n- 数据：抛地球仪9次，其中6次海面朝上。\n\n- 以二项分布函数作为**似然函数**。\n\n**我们该如何计算后验分布**"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"8B110CA7AB7846D7A701020EF7C29C6A","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"第一个方法是利用贝叶斯公式$p(\\theta|data) = \\frac{p(data|\\theta)p(\\theta)}{p(data)}$ 求出后验分布的解析解\n\n但是可能的困难在于\n\n1. 很多情况下求积分往往是很难的，并且有些积分可能没有解析解。\n$P(data) =\\int_{\\theta}^{} p(data,\\theta) =\\int_{\\theta}^{} p(data|\\theta)p(\\theta)d\\theta$\n\n\n2. 若后验分布比较复杂，即使获得后验分布，也难以计算期望值。\n$E(\\theta) =\\int_{\\theta}^{} \\theta p(\\theta|data) d\\theta$\n\n第二种方法：近似求解算法"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"EAB18BB0727743118FE1EDBF50FEC624","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"## Part 1: The idea of approximation"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"3BA8E3293FC24266834D765F3DB58C3C","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"### Grid approximation"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"C98702756B74464AA682A0C79C6BCEC7","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"最简单的近似方法就是网格近似。\n\n虽然大多数参数都是连续的，能够接受无穷多的值。\n\n但有时候通过有限的参数值，就可以很好地近似连续参数的后验分布"},{"cell_type":"code","execution_count":2,"metadata":{"collapsed":false,"id":"92E7836FF0244DC68D57990B845B15F1","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[],"source":"import numpy as np                             # numpy 是专门用于数组运算的包\nimport scipy.stats as st                       # 从scipy.stats里载入分布函数\nimport matplotlib.pyplot as plt                # matplotlib.pyplot 是专门用于画图的包\nimport seaborn as sns                          # seaborn是专门用于绘制统计数据的包\nimport arviz as az                             # arviz 是专门用于贝叶斯统计会绘制的包\nfrom scipy.stats import norm, uniform, binom   # 从scipy.stats里载入正态分布和均匀分布\nfrom math import sqrt                          # 从math包输入计算平方根的公式\nfrom matplotlib.patches import Circle          # matplotlib.patches是用来画几何图形的包\nsns.set_style(\"white\")                         # 选择一个seaborn的风格"},{"cell_type":"code","execution_count":3,"metadata":{"collapsed":false,"id":"ACA7876CAB8248608E58EE53D49E8695","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"data":{"text/plain":["[<matplotlib.lines.Line2D at 0x26b176ef610>]"]},"execution_count":3,"metadata":{},"output_type":"execute_result"},{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAXIAAAD6CAYAAAC8sMwIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAm4ElEQVR4nO3deXxU9b3/8VdWCAwSIggSDWIwKAhm4i7GDQMo1gWERCTgFVqtV29bccNWxJRCkNtef3WtpY0tLizKjUsVNIAEUBFDAkQhWJSIAQGFAAkhyWTO749DBrhAhmVmvrO8n4+HDzNzhpn3eQhvD2fO+XyjLMuyEBGRkBVtOoCIiJwcFbmISIhTkYuIhDgVuYhIiFORi4iEOBW5iEiIi/X2ArfbzcSJE6moqCA+Pp5JkybRrVs3z/bFixfz/PPPY1kWvXv35sknnyQqKsqvoUVE5ACvR+RFRUU0NDQwa9Ysxo0bR35+vmdbTU0N06ZN46WXXmLOnDkkJyezc+dOvwYWEZFDeT0iLykpITMzE4D09HTKy8s920pLS0lLS2Pq1Kls2rSJYcOGkZSU1OL7XXrppSQnJ59kbBGRyFJVVcXy5cuPuM1rkdfU1OBwODyPY2JicLlcxMbGsnPnTpYvX05hYSFt2rThzjvvJD09ne7dux/1/ZKTk5k7d+4J7IaISOQaMmTIUbd5PbXicDiora31PHa73cTG2v2fmJhInz596NSpE23btuWiiy5i7dq1PogsIiLHymuRZ2RkUFxcDEBZWRlpaWmebb1792b9+vXs2LEDl8vFqlWr6NGjh//SiojIYbyeWsnKymLZsmXk5ORgWRaTJ0+moKCAlJQU+vfvz7hx4xg7diwAgwYNOqToRUTE/7wWeXR0NHl5eYc8l5qa6vl58ODBDB482PfJRETkmOiGIBGREOf1iFxE5P8qLK1i2vwKNlfX0TUxgYcH9uRWpy4rNkVFLiLHpbC0ivFz11DX2ARAVXUd4+euAVCZG6JTKyJyXKbNryC6sYY7YhaQwD4A6hqbmDa/wnCyyKUjchE5Lm12fc0/4p+hR/RmEqnlxaabAdhcXWc4WeTSEbmIHLs1b/J2qwm0j6rh3+6uDIv5GLCX/e2amGAyWURTkYuId64G+OBReGsMdUnncbt7Ki81/Yyzo3/g4qgKEuJieHhgT9MpI5ZOrYhIy3ZVwZy74PvP4bL7ODUrj9+s3saz8xKp2fcP7mqzlMabcvRFp0EqchE5um8+hjfHgGsf3F4A59uDm251JtvF/c5wBq95E85rZzZnhNOpFRE5nNsNS/4IM26DNqfCzxd6SvwQzlHQuBe+/N/AZxQPFbmIHKpuJ8wcAQvyoPdtdol3Osr57zMugk7nQumMwGaUQ6jIReSALavh5Wvg3x/BDU/D0L9BK8fRXx8VBc6R8P0K2LYuYDHlUCpyEbGVvgp/y7KvUPmPD+DSe+yi9qZvDkTH6qjcIBW5SKRr3AfvPABv/yeceQncU2z/+1g5OkHaIFg10/6fgAScilwkku3cCH8fACv/CZnjILfQLubjlTEK9v4IX8/3dUI5BipykUi1fj785SrYsRHumAn9J0B0zIm9V2p/cHSBlTq9YoKKXCTSuJtg4SR4fTgkpsA9i6HnDSf3njGxkD7C/pJ09xbf5JRjpiIXiSS1P8KrQ6B4mn21yZiPIKm7b97bORIsN6x63TfvJ8dMRS4SKTatsE+lVH4KNz8LtzwPcT4cdHVqKnTrZ1/9Ylm+e1/xSkUuEu4sCz7/KxTcYF8mOOZD+8tJf3COhB3fQOUn/nl/OSIVuUg4a6iFuT+H9x+C1Ovs8+Fd0/33eb1ugfh29lG5BIyKXCRc/fg1/LU/lL8F1z1hX5mS0MG/nxnfFvoMha8KYd9u/36WeKjIRcLRl4X2rfa122DkXLjqIYgO0B93Z+7+QVpzA/N5oiIXCStNjTD/tzBnNJx2nn2XZuq1gc2QfCF0Ok/XlAeQilwkXOzeAv/4GXz6HFxyD9z1PrQ/I/A5mgdpVX0B29YG/vMjkIpcJBx8u8S+tHDLKnti4Y1PQ2y8uTwXNA/S0peegaAiFwlllgVLn4F/3gyt29uzw/vcbjoVtO1o3y266g0N0goAr0u9ud1uJk6cSEVFBfHx8UyaNIlu3bp5tk+aNImVK1fStm1bAF544QXatdOyTyJ+t28XFN4H696DXrfCLc9BqyD6s+ccBWvfhfXzoNfNptOENa9FXlRURENDA7NmzaKsrIz8/HxefPFFz/Yvv/yS6dOnk5SU5NegInKQH8phdi5UfwcDp8Blvzy22eGBlHodtDvdnlOuIvcrr6dWSkpKyMzMBCA9PZ3y8nLPNrfbTWVlJRMmTCAnJ4c333zTf0lFxFb2Bky/Hhr2wuj34PL7gq/E4aBBWkWwe7PpNGHNa5HX1NTgcBxY6ikmJgaXywXA3r17GTlyJNOmTWP69Om8/vrrrFun5Z5E/KJxH7z7ayi8114r894l0O1y06laln6nPUirTIO0/MlrkTscDmpraz2P3W43sbH2GZmEhARGjRpFQkICDoeDyy67TEUu4g87K6FgEJQUQL9f718A4jTTqbw7NRW6XalBWn7mtcgzMjIoLi4GoKysjLS0NM+2jRs3cscdd9DU1ERjYyMrV66kd+/e/ksrEom+LoKXr4afNkD2a5D1lH3aIlRk5MLOb6FymekkYcvr74asrCyWLVtGTk4OlmUxefJkCgoKSElJoX///txyyy0MHz6cuLg4brnlFs4555xA5BYJf+4mWPw0LJ4KnXvD8H/aR7ih5ryb4f2H7aPys640nSYsRVlWYP++M2TIEObO1QwGkRbV/mRPLdywAC64Awb/CeLbmE514t79tb0480MV9vXuctxa6k7dECQSbKpK7FMpG5fATc/ArS+GdomDPUjLVQflOojzBxW5SLCwLFjxN/j7ICAK7p4PF/1HcF5aeLySM+C0XvY15eJzKnKRYNCwF/73XvjXg9D9KnsBiOQM06l8xzNIqwS2fmU6TdhRkYuY9tMG+waf1bPgmsdhxBxoE4Z3SvfNhug4DdLyAxW5iElr37UXgNizGUa+Cdc8GrgFIAKteZDW6pkapOVjYfo7RiTINbngwydg1kg4tYe9AESP602n8r+MUbD3J1j/gekkYUVFLhJoe7baY2c/+TNcNAbungeJKaZTBUbqddCuq1YP8jEVuUggVX4Cf8mEqpVw21/gpj9BbCvTqQInOsYepLVhAeyqMp0mbKjIRQLBsuCT5+CVmyDeAT9fYK+iE4mc+wdprdIgLV9RkYv4277dMHsUfPhbOPdG+MXH9i33kSrpbDgr0756xe02nSYsqMhF/GnrV/DXa2Hdv2DAJBg+A1qfYjqVec5c2LlRg7R8REUu4i+rZ8P0/lC/B0a/C1c8EB53afrCeT+DVqfomnIfUZGL+JqrHv71kD306vR0+9LCs/qZThVc4tvYi0R/9ba99qicFBW5iC/t+h4KboQVf7WPwEe/A+26mE4VnJwj7UFaa7RE5MlSkYv4yoaF8FImbK+wz4UPmAQxcaZTBa+uGXBab51e8QEVucjJcrth8TSYMcQ++v7Fx1o1/lg0D9LavBK2fmk6TUhTkYucjL074I1sWDQJ+g6HsUXQsYfpVKFDg7R8QkUucqI2l9oLQGxYBIP/aN+pGd/WdKrQ0vZU+9r6VTPtL4nlhKjIRY6XZUHJP+BvA+3TKnfPh4vH6tLCE+UcBXU7oEKDtE6UilzkeDTWwdv3w7v/ZV9SeE8xnHGh6VShLfVaOCVZqwedBBW5yLHa8Q1Mz4KyV+HqR+HON+1TA3Jymgdp/XuBffmmHDcVucixWPc+/OUa2LXJXsHn2sftAhLfSL8TsKDsDdNJQpKKXKQlTS4oegpm3gFJ3e1TKWkDTKcKP0nd7UFaZRqkdSJU5CJHU7MdXr0Nlv4JLrzL/lKzQzfTqcJXxqj9g7SWmk4ScmJNBxAJSt99BnPugrqdcMsL9gxt8a/zfgat2tvXlHe/ynSakKIil4hXWFrFtPkVbK6uo2v71rx4zgr6fjUN2p9p3+DTpY/piJEhLsEepFX2GtzwNCQkmk4UMryeWnG73UyYMIHs7Gxyc3OprKw84mvGjh3LG2/oiwoJLYWlVYyfu4aq6jraUMf4vU/Tt3wKW07LtG+1V4kHlnMkuPZBuQZpHQ+vRV5UVERDQwOzZs1i3Lhx5OfnH/aaZ555ht27d/sloIg/TZtfQV1jE53YydvxT3BD9HLyG3MYtvN+HRGa0NUJnc/XLfvHyWuRl5SUkJmZCUB6ejrl5eWHbJ83bx5RUVGe14iEks3VdQCMj3uDM6O2M7LxcV5qupmqXbpd3AjPIK1S+KHc++sFOIYir6mpweFweB7HxMTgcrkAWL9+Pe+99x6/+tWv/JdQxI+6JiaQEbWeITFL+WvTjXzq7u15Xgzpmw0x8ToqPw5ev+x0OBzU1tZ6HrvdbmJj7V9WWFjI1q1bGT16NFVVVcTFxZGcnMxVV+kbZwkNDw84h7PffoitViIvuG4BICEuhocH9jScLIK1SYKeN8LqmZD1FMS2Mp0o6Hkt8oyMDBYtWsSNN95IWVkZaWlpnm2PPPKI5+dnn32Wjh07qsQlpNwavQSiNpAX9yvq6luTnJjAwwN7cqsz2XS0yJaRC18VQsX70Ps202mCntciz8rKYtmyZeTk5GBZFpMnT6agoICUlBT69+8fiIwi/lG/B4omQvJFTBgzkQnRuj8uaJx9LZxyBqycoSI/Bl6LPDo6mry8vEOeS01NPex1DzzwgO9SiQRC8X9DzVbIeR1U4sGleZBW8TR7kFb7M0wnCmr63SuR6acN8NkLcMEdcMZFptPIkTibB2m9bjpJ0FORS2T68An7yojrJ5pOIkfT4Sz7Vv1SDdLyRkUukWfDQqj4F2SOsxdLluDlHAXVlbBxiekkQU1FLpGlyQXzxttHe5fdZzqNeHPeTQcGaclRqcglsnzxd9i+Dgb8AeJam04j3sQlQN9hsPYdqKs2nSZoqcglcuzdAYv+AN2vhnMHm04jx6p5kNaaOaaTBC0VuUSORZOhfjcMyteK96Hk9HTo3EenV1qgIpfIsPVL+OJvcNEY6NzLdBo5Hs2DtLaUwQ9rTKcJSipyCX+WBfMeg1an2IsmS+jpO1yDtFqgIpfwt+49+LYYrv2tPZBJQk+bJDj3Jlg9C1waMfx/qcglvDXug/m/hU7nwUV3m04jJ8M50l5Ddd2/TCcJOipyCW+fPW/fUHJDPsRoidqQdvY19jqqpTNMJwk6KnIJX7u3QPEf7b+Sn32N6TRyspoHaW1YBNWbTKcJKipyCV8LngJ3Iwz4vekk4ivpGqR1JCpyCU/ffwGr3oDL/xOSzjadRnylQzf7hq4yDdI6mIpcwo/bDR88Co7O9mAsCS8Zo6D6O9hYbDpJ0FCRS/hZMxuqvrBH1LZqZzqN+Nq5g6G1BmkdTEUu4aV+D3z0JHTNgL45ptOIP8QlQJ/h8NU79uWIoiKXMLPkT1DzA9zwtJZvC2fOkdBUD2veNJ0kKOh3uoSPHd/Cp8/ZR+JnXmw6jfhT13To0kfXlO+nIpfw8eHvIDoOrn/SdBIJBOco2LIKtqw2ncQ4FbmEh28+tmeqZD4Ip3Q1nUYCoc/tENNKX3qiIpdw0Lx8W2IKXH6/6TQSKG2S7KXgVs+yZ+pEMBW5hL6SAtj2lZZvi0TOkbCv2l5MO4KpyCW0NS/fdlYmnPcz02kk0LpfYw/SWhnZX3qqyCW0fZwP+3Zp+bZIFR1tz1/55mP7bs8I5bXI3W43EyZMIDs7m9zcXCorKw/Z/tprrzF06FBuv/123n//fb8FFTnM1q9gxXS48D+gy/mm04gpzjvtf0fwIC2vRV5UVERDQwOzZs1i3Lhx5Ofne7bt2LGDN954g5kzZ/LKK68wdepULMvya2AR4KDl2xz2yj8SuRJT4OyrofS1iB2k5bXIS0pKyMzMBCA9PZ3y8nLPtqSkJAoLC4mLi+PHH3+kVatWROmvtxIIFe/Dt4vtEm97quk0YpozF3Z9Z/+eiEBei7ympgaHw+F5HBMTg8vl8jyOjY3l1VdfJTs7m5tvvtk/KUUO5qqH+Y9Dp3O1fJvYzr0JWidG7J2eXovc4XBQW1vreex2u4mNPXTJrJEjR7JkyRJWrFjBZ5995vuUIgf77AXYuREGTYGYONNpJBjEtYa+w2Hte/aVTBHGa5FnZGRQXGzP/S0rKyMtLc2z7ZtvvuH+++/Hsizi4uKIj48nWoOKxJ/2/ADF/w09b4TU60ynkWDizI3YQVpeV6PNyspi2bJl5OTkYFkWkydPpqCggJSUFPr378+5555LdnY2UVFRZGZmcskllwQit0SqBXn2qZUBk0wnkWBzel/o0tc+vXLpL0ynCSivRR4dHU1eXt4hz6Wmpnp+vv/++7n/ft0WLQFQVQJlr0G/X8Gpqd5fL5EnYxS8/5A9TOv0C0ynCRidB5HQYFn28m1tT4PMh0ynkWAVoYO0VOQSGlbPhu9X2CNqW59iOo0Eq4QO9qiG1bMjapCWilyCX30NFD0JXZ1wwQjTaSTYNQ/SWvee6SQBoyKX4Lf0f2DPFi3fJsem+9XQPiWirinXnwoJbjs3wifP2ovtnqkrouQYREfb81e+WQw7K72/PgyoyCW4ffgERMfA9RNNJ5FQkr7/FFyEDNJSkUvw+rYY1r4DVz4I7ZNNp5FQkpgCZ19jX64aAYO0VOQSnJqXb2ufAlfoPgU5ARm5sGsTfPux6SR+pyKX4LTyH7C1HAb8HuISTKeRUHTuTfbliBGwepCKXIJP3U5YOAm6XQm9bjGdRkJVbCv7S/J14T9IS0UuwefjqfZ1wIOmaPk2OTkZudDUAGvmmE7iVypyCS7b1sHnL0PGaHsIksjJ6NLHnrkS5teUq8gleBy8fNt1vzOdRsKFMxd+WAOby0wn8RsVuQSP9fPgm0VwzXho29F0GgkXETBIS0UuwaF5+baOaXDxWNNpJJwkdIBeN8Oa2dBYZzqNX6jIJTgsfwl2fKPl28Q/nCNh3y5Y9y/TSfxCRS7m7dkKi6dB2iDocb3pNBKOzrrKvttz5T9NJ/ELFbmYtzAPXPtgwB9MJ5FwFR0N6SPh2/AcpKUiF7OqVkLpa3DZvdCxh+k0Es7SRwBR9vyVMKMiF3OaLzds2xGueth0Ggl3iWdC6rX2gYO7yXQan1KRizlr3oRNy6H/BGjd3nQaiQTOXNj9PXzzsekkPqUiFzMaauGjCXB6un3uUiQQzh1sX44YZnd6qsjFjKXPwJ7NcMNULd8mgRPbCvpm25chhtEgLf0JksDbWQmf/BnOvx1SLjOdRiKNc6Q9SGv1bNNJfEZFLoH30QQgCrKeMp1EIlGXPvYpvdIZ9hfuYUBFLoG1cSl8VQhX/gban2E6jUSqjFx74ZItZaaT+ISKXALH3QQfPAbtz4QrHjCdRiLZ+bdDbOuwGaTltcjdbjcTJkwgOzub3NxcKisPvSvqlVdeYdiwYQwbNoznnnvOb0ElDKz8J2xdA1l5EN/GdBqJZAmJcN7NsHpOWAzS8lrkRUVFNDQ0MGvWLMaNG0d+fr5n26ZNm3jnnXeYOXMms2fPZunSpaxbt86vgSVE1VXDwt9DyhXQ+zbTaUTsLz3rd8Ha90wnOWlei7ykpITMzEwA0tPTKS8v92zr0qUL06dPJyYmhqioKFwuF61atfJfWgldi6fal3vdkK/l2yQ4nJUJid2gNPQHaXkt8pqaGhwOh+dxTEwMLpcLgLi4OJKSkrAsi6lTp9KrVy+6d+/uv7QSmrZX2Mu3XTjaXnZLJBhER9tH5d8Ww86NptOcFK9F7nA4qK2t9Tx2u93ExsZ6HtfX1/PQQw9RW1vLk08+6Z+UErosC+aNh7i2cN0TptOIHKp5kFZpaA/S8lrkGRkZFBcXA1BWVkZaWppnm2VZ3HffffTs2ZO8vDxiYmL8l1RC09cfwoYFcM2jWr5Ngk/7M6BHfyh7PaQHacV6e0FWVhbLli0jJycHy7KYPHkyBQUFpKSk4Ha7+fzzz2loaGDJkiUAPPjggzidTr8HlxDgarCPxk89By7+uek0IkfmHAlz7rLXiw3RhU28Fnl0dDR5eXmHPJeamur5ec2aNb5PJeHh87/Ajg1w55sQG286jciR9bwREpJg5YyQLXLdECT+UbMNFj8N5wyAc7JMpxE5uoMHadX+ZDrNCVGRi38s/D007oWBk00nEfEuIxfcjbAmNAdpqcjF9zaX2X9NvfRe6HiO6TQi3nXuDV2d9u/bEBykpSIX37Is+OBRaHOqlm+T0OLMhW1fwuZS00mOm4pcfKv8Ldj0mb18W0Ki6TQix+78oSE7SEtFLr7TsNeeNd6lr31Jl0goSUiEXrfYa8mG2CAtFbn4zrL/B7ur9i/fppvDJAR5Bmm9azrJcVGRi29Ub4Jlz0DvIdDtCtNpRE5Mtyuhw1n2yOUQoiIX3/hogv3vrLyWXycSzJoHaW1cAju+NZ3mmKnI5eRVfgJfzoV+v4bEM02nETk5F4yAqGgoC51BWipyOTnuJvtyw1OSod+vTKcROXntkyE1tAZpqcjl5JS+Cj+s1vJtEl6cI+0v7jcsMp3kmKjI5cTVVcOCPEi53L4GVyRc9LzRvqktRFYPUpHLiSueBnt/gkFavk3CTGz8/kFa74fEIC0VuZyYH7+G5S/Zw4a6pptOI+J7zv2DtFbPMp3EKxW5nJj5j0NcGy3fJuGrcy/ommF/DxTkg7RU5HL81n9oL+F29SPgOM10GhH/yWgepLXSdJIWqcjl+Lga7KPxpFS45B7TaUT86/yhEJsQ9IO0VORyfFb8FX76GgZN0fJtEv5atz8wSKthr+k0R6Uil2NXsx0+nmqva3jOANNpRAIjIxfqdwf1IC0VuRy7RZOgsRYGTtHlhhI5uvWDDt2hdIbpJEelIpdjs2UVlPzDPi/eKc10GpHAiYo6aJDWN6bTHJGKXLyzLPjgMWiTZF+pIhJp0vcP0ioNzkFaKnLx7sv/he8+sa8Z1/JtEolO6Wp/NxSkg7RU5NKy5uXbOveBjFGm04iY4xwJezbDhoWmkxxGRS4t++RZ2LUJbsjX8m0S2dJusAdpBeHqQV6L3O12M2HCBLKzs8nNzaWysvKw1+zYsYOBAwdSX1/vl5BiyK7vYen/QK9b4awrTacRMSs2HvrmQMUHUPuj6TSH8FrkRUVFNDQ0MGvWLMaNG0d+fv4h25csWcLdd9/N9u3b/RZSDPnoScDS8m0izZwjg3KQltciLykpITMzE4D09HTKy8sPfYPoaAoKCkhMTPRLQDGk8lMofxOu+C/o0M10GpHg0LkXJF8IK2cE1SAtr0VeU1ODw+HwPI6JicHlcnke9+vXjw4dOvgnnZjhdsO8/cu3Xflr02lEgoszF7avhargGaTltcgdDge1tbWex263m9jYWL+GEsPKXrVvAMrKg/i2ptOIBBfPIK3gudPTa5FnZGRQXFwMQFlZGWlpuqsvrO3bZS/fduZlWr5N5EhanwK9b4Xyt4JmkJbXIs/KyiI+Pp6cnBymTJnC+PHjKSgoYMGCBYHIJ4FWPM3+Rv4GLd8mclTO5kFa75hOAoDXcyTR0dHk5R161UJqauphr1u4MPgukpfj9OO/4bOXwHkndHWaTiMSvLpdAUln2196XpBjOo1uCJKDfPhbiG0N100wnUQkuDUP0qpcCj9tMJ1GRS77fV0E6+fB1Q9Du86m04gEvwvusAdplZkfpKUiF2hqhPnj7b8qXnqv6TQioeGUrtAjyx6k1eTy/no/UpELrJgOP66HgZMhtpXpNCKhwzkS9mwxPkhLRR7pan+ERVMg9TpIG2Q6jUhoSRsEbTpCqdlBWirySLdwEjTUwCBdbihy3GLj7atWDA/SUpFHsi2roeQVuOQX0Kmn6TQiocmZC24XrJppLIKKPFJZFswbDwkd4JpHTacRCV2nnQtnXGzfsm9okJaKPFJ99bZ9Dex1v7PLXEROnHMkbF8HVSVGPl5FHoka6+DDJ6Dz+XDhXabTiIS+3kMgro2xQVoq8kj0yXOw6zsYNEXLt4n4QutT7JW01rwFDbVeX+5rKvJIs6sKlv4JzrsZul9lOo1I+MjIhYY98FXgB2mpyCNN0URwN8GA35tOIhJeUi6HpFQjp1dU5JHku+WwZjZc8QB0OMt0GpHw4hmktSzgg7RU5JGiefm2dl0h80HTaUTCU/MgrdJXA/qxKvJIsep12FwKWU9p+TYRfznldDhnQMAHaanII8G+3VD0FJxxCfQZZjqNSHhzjoSaH2BD4FZRU5FHgiX/DbXbtHybSCCkDYK2nWBl4AZpqcjD3U8b4NMXIP1OSL7QdBqR8BcTB32z7YVaarYH5CNV5OHuw9/ZM8b7a/k2kYDJGGUP0lodmEFaKvIwVFhaRb/8hYx6fApUvM+XPX4B7bqYjiUSOTr1tL+TKn01IIO0VORhprC0ivFzV7O3eiu/i53BRndn7liTQWFpleloIpGleZDW91/4/aNi/f4J4juuBvtLyz1boWar/c2452f7n8uqKlkVvZP41k0A/LzhQXa7o5k2v4JbncmGd0Akgpw/BOY9Zt/peebFfv0oFblplgX1e+wi3vPDIaVsl/QPULPN3la34whvEAVtTrVPnTg6s7SpLdusRLZZiVRYZ/KpuzcAm6vrArtfIpGuVTvofRuUz7UH1Pnx/g0Vub+4m+ylnw4u4kN+3nbgiNp1hJKNiQdHF2jX2V7dPuXy/WV9mv284zT7cdtO9rfk+/1P/kKqjlDaXRMT/Lm3InIkzlwoe82e/58+wm8foyI/Xo11hxfxkU5z1G4Hy334r2/d/kARn3ExODrb/xxc0u06Q+vEE7rm++GBPRk/dw11jU2e5xLiYnh4oJZyEwm4lMvg1B6wcobZIne73UycOJGKigri4+OZNGkS3bp182yfPXs2M2fOJDY2ll/+8pdce+21fgvrN5YFdTu9n9qo2Qr1uw//9VEx+0v4NGh3Opx+gedUx4GS7mxvj/PvkXHzefBp8yvYXF1H18QEHh7YU+fHRUxoHqRVNBF+/Dd07OGXj/Fa5EVFRTQ0NDBr1izKysrIz8/nxRdfBGD79u3MmDGDt956i/r6ekaMGEG/fv2Ij4/3acjC0qoTK6bmLwePVsqe57eCu/HwXx/X5kARd+4FqdfZR8uOzgeOnB2d7XPUQbRAw63OZBW3SLC44A7cC37Pay/+gQm1t/vl4MprkZeUlJCZmQlAeno65eXlnm2rV6/G6XQSHx9PfHw8KSkprFu3jr59+/osoH053YFTBVXVe5k0dzmOPadxfQpHOc2xtYUvB7GLt/n0Rse0g46c9xd088+t2vlsP0QkMhX+u4lT3OkMcC9kIrdRVV3H+LlrAHxW5l6LvKamBofD4XkcExODy+UiNjaWmpoa2rU7UHZt27alpqbGJ8GaTZtfQV1jE32ivuGZuOc5PWoHbaLqYeH/eWFM/IHTFx262+emjnT+2XHaIV8Oioj407T5FfRuvIqX40u4LPorlrn7UNfY5NNLgr0WucPhoLb2wBp0breb2NjYI26rra09pNh9ofmyud20ocSdxm7asM1K5EcrkT+NGXjg/HNCBw2EEpGgs7m6jq04ecY1hG/dpx/yvK94LfKMjAwWLVrEjTfeSFlZGWlpaZ5tffv25ZlnnqG+vp6GhgY2bNhwyHZf6JqYQFV1HZVWFx5x3eN5PjkxAVJD8ItVEYkozR32jOv2w573Fa+36GdlZREfH09OTg5Tpkxh/PjxFBQUsGDBAjp16kRubi4jRoxg9OjR/OY3v6FVq1Y+Cwf25XQJcYd+kajL6UQkVASiw6IsKwATXQ4yZMgQ5s6de1y/5oSvWhERCQK+6LCWujMkbgjS5XQiEsr83WGafigiEuJU5CIiIU5FLiIS4lTkIiIhTkUuIhLiAn7VSlVVFUOGDAn0x4qIhLSqqqMv1xjw68hFRMS3dGpFRCTEqchFREKcilxEJMSpyEVEQpyKXEQkxAVlkbvdbiZMmEB2dja5ublUVlYesn327NkMGTKE4cOHs2jRIkMpfcvbPr/yyisMGzaMYcOG8dxzzxlK6Tve9rf5NWPHjuWNN94wkND3vO3z4sWLGT58OMOGDWPixImEwwVl3vb573//O0OGDGHo0KF89NFHhlL6x6pVq8jNzT3s+YULFzJ06FCys7OZPXu2bz7MCkLz58+3Hn30UcuyLKu0tNS69957Pdu2bdtm3XTTTVZ9fb21e/duz8+hrqV9/u6776zbbrvNcrlcltvttrKzs621a9eaiuoTLe1vsz/+8Y/WsGHDrNdffz3Q8fyipX3es2ePNXjwYOunn36yLMuyXn75Zc/Poaylfd61a5d19dVXW/X19VZ1dbV1zTXXmIrpcy+//LJ10003WcOGDTvk+YaGBuv666+3qqurrfr6emvIkCHW9u3bT/rzgvKI/FgXfG7Xrp1nwedQ19I+d+nShenTpxMTE0NUVBQul8vnC3gEWkv7CzBv3jyioqI8rwkHLe1zaWkpaWlpTJ06lREjRtCxY0eSkpJMRfWZlvY5ISGBrl27UldXR11dHVFhtFRjSkoKzz777GHPb9iwgZSUFNq3b098fDwXXnghK1asOOnPC8p55KYXfDahpX2Oi4sjKSkJy7J4+umn6dWrF927dzeY9uS1tL/r16/nvffe489//jPPP/+8wZS+1dI+79y5k+XLl1NYWEibNm248847SU9PD+v/zgCnn346gwcPpqmpiXvuuedobxNyBg4cyPfff3/Y8/7qr6AsctMLPpvQ0j4D1NfX8/jjj9O2bVuefPJJExF9qqX9LSwsZOvWrYwePZqqqiri4uJITk7mqquuMhXXJ1ra58TERPr06UOnTp0AuOiii1i7dm3IF3lL+1xcXMy2bdtYsGABAGPGjCEjI4O+ffsayRoI/uqvoDy1kpGRQXFxMcARF3wuKSmhvr6ePXv2+GXBZxNa2mfLsrjvvvvo2bMneXl5xMTEHO1tQkZL+/vII48wZ84cZsyYwW233cZdd90V8iUOLe9z7969Wb9+PTt27MDlcrFq1Sp69OhhKqrPtLTP7du3p3Xr1sTHx9OqVSvatWvH7t27TUUNiNTUVCorK6murqahoYEvvvgCp9N50u8blEfkWVlZLFu2jJycHCzLYvLkyRQUFJCSkkL//v09Cz5bluWXBZ9NaGmf3W43n3/+OQ0NDSxZsgSABx980Ce/AUzx9t84HHnb53HjxjF27FgABg0aFBYHKN72+ZNPPmH48OFER0eTkZFBv379TEf2i3fffZe9e/eSnZ3NY489xpgxY7Asi6FDh9K5c+eTfn8NzRIRCXFBeWpFRESOnYpcRCTEqchFREKcilxEJMSpyEVEQpyKXEQkxKnIRURCnIpcRCTE/X/MTfwSaLzQ7wAAAABJRU5ErkJggg==","text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":"p_grid = np.linspace(0,1,5)        # 取5个离散的参数值\nprior = np.repeat(1,5)             # 5个参数值的不标准的先验分布\nlikelihood = binom.pmf(6,9,p_grid) # 投掷9次，6次海面朝上的各参数的似然\nunstd_posterior = likelihood*prior #没有归一化的后验分布\n\nposterior = unstd_posterior/np.sum(unstd_posterior) # 后验分布归一化\n\nplt.plot(p_grid,posterior,'o') #画出5个值的点\nplt.plot(p_grid,posterior) #将5个值得点进行连线"},{"cell_type":"code","execution_count":4,"metadata":{"collapsed":false,"id":"C83F0EC4A7364670A95C24E6B194731F","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"data":{"text/plain":["[<matplotlib.lines.Line2D at 0x26b197f0100>]"]},"execution_count":4,"metadata":{},"output_type":"execute_result"},{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAXcAAAD3CAYAAADmBxSSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAuuElEQVR4nO3de1xUdf7H8dcwA4gMipaXpDBTQS0NsLIidRORzNZsXIVINrfLWrZtq/7M3C1z21LK1e6XdTNqLRVLluymhZoo5o1AJRVNCwtNSUUZRIaZOb8/jo6xKeNlhu/M8Hk+Hj1wzmHgfUzfHs75nu/XoGmahhBCiIASpDqAEEIIz5NyF0KIACTlLoQQAUjKXQghApCUuxBCBCCT6gAn9enTh6ioKNUxhBDCr5SXl7Nu3bpfbfeZco+KiiInJ0d1DCGE8CsWi+W02+WyjBBCBCApdyGECEBS7kIIEYCk3IUQIgBJuQshRADymdEyQojGlVtUzoylpeytrKFDZBgTU2IZFi/DkQOFlLsQTVBuUTmTc7ZQU+cAoLyyhsk5WwCk4AOEXJYRogn655JtRNnLGGH8kiFBawGoqXMwY2mp2mDCY+TMXYimoPoglG+EHzfCjxv49PhaWoTWuHYH2+zkOm9ib2VNA19E+BMpdyH81BmvmdttsL9EL/Jyvcw5tFt/kyEI2l3JMlM/Co5fziZnZ54yvc2zwbPZY2vL/pZXqz0o4TFuy93pdDJ16lRKS0sJCQnh6aefpmPHjvU+59ChQ9x5550sXryY0NBQ1/Zdu3YxcuRI1qxZU2+7EOLCnLpmbucSDnH10Z0c/u+bHFy1j4uObgX7cf0Tze3h0msg4W649FroEAch4RiKyvkkZws1DgcP1j1CbsgUZoc8T2HfRUqPS3iO23LPy8vDZrORnZ1NcXExmZmZvP766679q1atYubMmVRUVNR7n9Vq5dlnnyUkJMTzqYVo4mYsLaWmzsHfTW9zt+kLAI5rwZQe7sxF19+nF/ql10KLKDAYfvX+kzdN9TN/+Guzv/GO86+kbB4H1yyBUHNjHo7wArflXlhYSN++fQGIi4ujpKSk3v6goCCysrIYPny4a5umaTzxxBOMHz+esWPHejiyEGJvZQ1DgtZyt+kLFth/w3uOgWzXorFj4ruUIWf1NYbFR9UfGbMzCuaNgP+OgZFzIUjGW/gzt//3rFYrZvOpf8WNRiN2u931OjExkVatWtV7zyuvvEL//v3p1q2bB6MKIU6Kb1nF9OA3KXJ24XH7PWzRrqAOEx0iw87/i3YdCCnTYPvHsPwfngsrlHBb7mazmerqatdrp9OJydTwCf/ixYtZtGgRGRkZVFRUcM8991x4UiGEzmHn3+H/IgiNP9c9hP3ED+BhwUYmpsRe2Nfu8wD0Hg2rZ8Gm7AvPKpRxe1kmISGBFStWcOutt1JcXExMTIzbL/rFF1+4fj1gwADeeuutC0sphDglfwYXHfqajb2fxbn1cgyefMLUYIBb/wkHd8Hih6F1J7jsOs/kFo3KbbknJydTUFBAWloamqYxbdo0srKyiI6OJikpqTEyCiFOKvsK8p+DXmlcM/QBCoZ64XsYg2Hkf+DfA2BBOty/AiIv88I3Et5k0DRNUx0C9NVEZCUmIRpQcxje6AtBRhizCpq18O73qyiFNwdCZEe4R0bQ+KozdafcDhfCH2gafPQXqNoHw9/yfrEDtImF32XBgW/0ETROp/e/p/AYKXch/EHRu7A1F27+G1zau/G+b9eBkDJdRtD4IZl+QAhf9/NO+OxRuLwvJD7S+N+/zxio2KaPoGnTDa5ObfwM4pzJmbsQvsxeCx/cA6ZQsMzWr7c3tpMjaC7vq4+g+WF942cQ50zKXQhftuwp+Gkz3P4qtOigLsfJETQtOugjaCp/UJdFnBUpdyF81bd58NUrcM290O3sphTwquatIT1b/2li/p1Qa1WdSDRArrkL4YusFfDfB/Vr3CnPqE5zyskRNPNGsDfr94w8PJbyI7WyTJ8PkjN3IXyNpsGHY+H4EfjdWxB8AfPFeEPXgWy+chIdflpGWvV/0Di1TF9uUbnqdOIEKXchfM26N2Dn5zDoaWh3peo0p/XgzmuZZx/An0wfMixoNSDL9PkaKXchfMlPW+CLKRBzC1x3v+o0Z7T3yHGm2Eez1tmdp4Pfwswxfbss0+czpNyF8BW2Y/qwx7DW+uiY0yyy4Ss6RIZhx8T0ujsxG44z1PiVa7vwDVLuQviKpZPh5x1wxxsQfrHqNA2amBJLWLCRTVpntjo7cqdxmWemHBYeI+UuhC/YuhgK34Yb/wydb1adxq1h8VFMt/QkKrI58x0D6Bn0Pa8NCJLRMj5EhkIKoUhuUTkzlpaiVf7IkmaTcUReRasBT6iOddZcy/Qd7w3/XMDN1k+AQapjiRPkzF0IBXKLypmcs4V9ldXMCnkNo1bHnQfvI3dLhfs3+5pmLeGq4bDlA6itUp1GnCDlLoQCM5aWUlPn4D7jJ1wftI0n7aPZXtfWf4cS9h4NNiuULFKdRJwg5S6EAnsra2jOccaaFrPCcTUfOPq5tvulS6+Btlfq9w2ET5ByF0KBDpFh3GlcRqShmpfsFsDg2u6XDAb97H1vEezbpDqNQMpdCCUmDezEH02f8pWjB0VaVwD/H0rYawSYmkHhO6qTCM6i3J1OJ1OmTCE1NZWMjAzKysp+9TmHDh0iJSWF2tpaAKqqqnjggQcYNWoUqampFBUVeT65EH5sqGEl7QyHyW42AgMQFRnGdEtP/x5KGNYKrrwDNi8EW7XqNE2e26GQeXl52Gw2srOzKS4uJjMzk9dff921f9WqVcycOZOKilN3+bOysrj++usZPXo0u3fvZsKECfz3v//1zhEI4W8cdlj9AlwSxwt/HMcLPvwk6jnrPRo2zYeSHEjIUJ2mSXN75l5YWEjfvn0BiIuLo6SkpP4XCAoiKyuLyMhI17bRo0eTlpYGgMPhIDQ01IORhfBzW3Ph8HfQd4JPTzFwXi7ro09TLDdWlXNb7larFbPZ7HptNBqx2+2u14mJibRq1aree1q0aEGzZs2oqKhg4sSJjB8/3oORhfBjmgarn4eLY6DbbarTeN7JG6vlG/VJ0IQybsvdbDZTXX3q+pnT6cRkcv9ga2lpKaNHj2bcuHFcd911F5ZSiECx83PYXwI3jYOgAB3P0CsVjKFyY1Uxt3+6EhISyM/PB6C4uJiYmBi3X/Tbb7/lkUceYebMmfTv3//CUwoRCDQNVs2ElpdBzxGq03hP89Zw5TDYnK3PdCmUcHsKnpycTEFBAWlpaWiaxrRp08jKyiI6OpqkpKTTvmfmzJnYbDaeeUZfHsxsNte7CStEk1S2Bn5YB4Nn6AtOB7Leo/Vy/+a/EH+X6jRNkkHTNE11CACLxUJOTo7qGEJ4z7vDYW8x/GULhDRXnca7NA1evQ6aRcJ9X6hOE9DO1J0BetFPCB+ztxi+zYMbxgZ+scOpG6s/rof936hO0yRJuQvRGFY/D6Et4Nr7VCdpPFffCcYQubGqiJS7EN72807Y+qFe7M1aqk7TeJq3hu5DYfMCubGqgJS7EN5W8AKYQuH6saqTNL7eo+H4Ef0fN9GopNyF8KYjP8KmBZDwezC3UZ2m8V1+E7TuLE+sKiDlLoQ3rXlF/3jjw2pzqHLyxuoPa+HANtVpmhQpdyG8pfpn/Yy150iIjFadRp24dAgKlhurjUzKXQhvWfcG2I/DTX9RnUSt8Iuh+2/12SLr/HSlKT8k5S6ENxw/CutnQ/fboI0fL8DhKb1Hw/FK2LpYdZImQ8pdCG/Y+JY+SuQmmREVgMv7Qusr4Gu5NNNYpNyF8LS64/DVq3DFzRCVoDqNbwgKgoS7oawAKnaoTtMkSLkL4WnF70L1AegrZ+31xN2l31iVs/dGIeUuhCc57FDwIlx6rX4pQpxibgPdhkDxPP2nG+FVUu5CeFLJIqjco19rD7Ql9Dyh991Qcwi2f6w6ScCTchfCU5xOfYKwtj0g5hbVaXxTp99AZEd5YrURSLkL4Sk7PoOKbYG9hN6FCgrSz96/XwU/f6s6TUCTP4FCeMLJJfQiO8KVFtVpfFvcKAgywddvq04S0KTchfCE7/KhvBASHwGj+wXkm7SIdhA7WL+xaq9VnSZgSbkL4QmrZ4G5nT7cT7jXezQcOyg3Vr1Iyl2IC1VeCLu/hBseguBmqtP4hysGQMtoubHqRW7L3el0MmXKFFJTU8nIyKCsrOxXn3Po0CFSUlKordV/xDp+/DgPP/ww6enp3H///Rw6dMjzyYVQLLeonMTM5Sx5YxJHCefj4MGqI/mPoCC2XjIMvsvn5slvkpi5nNyictWpAorbcs/Ly8Nms5Gdnc2ECRPIzMyst3/VqlXcc889VFRUuLbNnz+fmJgY5s2bx7Bhw3jttdc8n1wIhXKLypmcs4WwIzu5xbiBLPsgJn60WwrqLOUWlfPAN92xa0GkGr+kvLKGyTlb5PfPg9yWe2FhIX376k/axcXFUVJSUv8LBAWRlZVFZGTkad/Tr18/vvrqKw9GFkK9GUtLqalzMMb4Mce0UN62p1BT52DG0lLV0fzCjKWl7KlryTJnAsONKwnCKb9/Hua23K1WK2az2fXaaDRit9tdrxMTE2nVqtWv3hMREQFAeHg4VVVVnsorhE/YW1nDRRxhqHENHzj6cZgWru3CvZO/Tx85bqCN4SgJhh31tosL57bczWYz1dXVrtdOpxOTqeGhXr98T3V1NS1atLjAmEL4lg6RYaQZVxBqsPOOY1C97cK9k79PK51XY9OMDDR+XW+7uHBuyz0hIYH8/HwAiouLiYmJcftFExISWLlyJQD5+fn07t37AmMK4VseTb6CDFMe+Y6e7NKiAAgLNjIxRRbmOBsTU2IJCzZSRXPWOnuQHFQov38e5rbck5OTCQkJIS0tjenTpzN58mSysrJYtmzZGd9z5513snPnTu68806ys7P505/+5NHQQqh2e+jXtDcc4qNmv8UAREWGMd3Sk2HxUaqj+YVh8VFMt/QkKjKMPGdvOgft4+Vks/z+eZBB0zRNdQgAi8VCTk6O6hhCnJ23boGqffDw1xBkVJ3Gv1X+AC9cBQP/LuvNnoczdac8xCTEudq3CfZ8BdfeL8XuCZGXQfueUPqZ6iQBRcpdiHO1bjYEN4f4UaqTBI7YIfDDOqj+WXWSgCHlLsS5qP4ZtrwPV6dBWKTqNIEjdjCgwY4lqpMEDCl3Ic7F1++AoxauG6M6SWC55GpoESWXZjxIyl2Is+Www4Y50Kk/tO2mOk1gMRj0s/ddy6FOHmTyBCl3Ic7W9o/haDn0eUB1ksAUOxjqjsHulaqTBAQpdyHO1vrZEBkNMSmqkwSmy/tCSASUfqo6SUCQchfibPy0BcoK4Lo/yvBHbzGFQpck/aaq06k6jd+TchfibKz7lwx/bAyxt4J1P+z9WnUSvyflLoQ7xw7pwx97pUJYK/efL85f12QwGOXSjAdIuQvhztfvgP24fklGeFfz1tDxRtgu5X6hpNyFaIjDDuvfhE79oF0P1WmahtjBULENDu1WncSvSbkL0ZDST+Hoj/LQUmOKPbEWbak8rXohpNyFaMi6f0HL6FOFI7yv9RXQprtcd79AUu5CnMlPJVC2Gq67T4Y/NrbYwVC2Rr+ZLc6LlLsQZ7L+X2AKg/gM1Umanm5DQHPAt3mqk/gtKXchTufYIdj8PvQaqY/gEI2rQwKEt4Xtn6hO4rek3IU4na//A/Ya6CM3UpUICoLYW+DbZWCvVZ3GL0m5C/G/HHbY8KY+10m7K1Wnabpih4CtCr5frTqJX5JyF+J/7fgMjvwgDy2pdkV/fcoHGTVzXtyWu9PpZMqUKaSmppKRkUFZWVm9/QsXLsRisTBy5EhWrFgBwN69exk1ahR33XUXY8eOpaZG5mcWfmTdv6DlZfo8J0Kd4DDoPEBfwEPTVKfxO27LPS8vD5vNRnZ2NhMmTCAzM9O1r6Kigrlz57JgwQLmzJnDrFmzsNlsvP322wwePJj33nuPrl278sEHH3j1IITwmP3fwPer4Np7wWhSnUbEDtbn0N+3SXUSv+O23AsLC+nbty8AcXFxlJSUuPZt3ryZ+Ph4QkJCiIiIIDo6mu3bt9O9e3eOHj0KgNVqxWSSvyTCT6yfDaZmkHC36iQCoGsKYJDl986D23K3Wq2YzWbXa6PRiN1ud+2LiIhw7QsPD8dqtdK+fXvee+89hgwZQn5+PrfccosXogvhYTWHYVM29Bwhwx99hbkNXNZHrrufB7flbjabqa6udr12Op2uM/H/3VddXU1ERATPPfcc06dP55NPPuFvf/sbkyZN8kJ0ITzs67ky/NEXxQ6GnzZD5Q+qk/gVt+WekJBAfn4+AMXFxcTExLj29erVi8LCQmpra6mqqmLXrl3ExMTQokUL1xl927ZtXZdohPBZTgds+Dd0TIT2PVWnEb908sb2DplI7Fy4vRienJxMQUEBaWlpaJrGtGnTyMrKIjo6mqSkJDIyMkhPT0fTNMaNG0doaChPPPEETz31FE6nE03TmDJlSmMcixDnb8cSqNwDg55WnUT8rzYxcFEX/dLMdferTuM3DJrmG2OMLBYLOTk5qmOIpuqdoXBwFzyySUbJ+KLPH4e1b8Cju6BZS9VpfMqZulMeYhLiwDb4bqUMf/RlsbeCs06fjkCcFSl30WTlFpWTmLmcd1/6G7UE82nwINWRxJlc1gfCWsuomXMg5S6apNyicibnbKGqsgKLcTW59kQmfPIjuUXlqqOJ0wkyQswtsPNzcNSpTuMXpNxFkzRjaSk1dQ5GGFfS3FDLO45B1NQ5mLG0VHU0cSbdboXjR2DPV6qT+AUpd9Ek7a2swYSdP5iWss7Zja3a5a7twkddcTMYQ2G7XJo5G1LuoknqEBnGbUFrudTwM/+y31Zvu/BRoWZ9psjST2UisbMg5S6apImDYngw+GNKnZeywhkHQFiwkYkpsWqDiYbF3gqVZfoIJ9EgKXfRJA2L2EasYQ/vh94BBBEVGcZ0S0+GxUepjiYaEnNinqpSWX7PHRnUK5qm1S9Aiyge//MUHjeFqE4jzlaLS/T1VUs/g34TVafxaXLmLpqeHzdC2Wq4fixIsfufbrdCeSFU/aQ6iU+TchdNT8EL+iPsvWXOdr90ciIxmeO9QVLuomn5+VvY9jFcez+ERrj/fOF72vaAyGgpdzek3EXTsuYlMIbInO3+zGCA2CGw+0uotapO47Ok3EXTUbUfNs2H+LvA3FZ1GnEhYgeDoxZ2r1CdxGdJuYumY93r4LTDDX9SnURcqI436vdN5NLMGUm5i6bh+FHY8BZ0HwoXdVadRlwoYzB0HaQvsuJ0qE7jk6TcRdNQ+DbUHoHER1QnER6yIfR6OHaQEX97gcTM5TKj5/+QcheBz26Dta9Bp34QlaA6jfCA3KJyxq5rjU0zkmT8mvLKGibnbJGC/wUpdxH4tiyEqn1y1h5AZiwtpaIulLXOHiQHFQKaTNn8P9yWu9PpZMqUKaSmppKRkUFZWVm9/QsXLsRisTBy5EhWrNDvXB87doxHH32U9PR0RowYwebNm72TXgh3nE4oeAna9YTOSarTCA85OTXzF87edA7aR1dDeb3t4izKPS8vD5vNRnZ2NhMmTCAzM9O1r6Kigrlz57JgwQLmzJnDrFmzsNlszJkzh65duzJv3jz+8Y9/sHv3bq8ehBBntGMJ/Fyqn7UbDKrTCA85OTXzUse1ODUDg4PW19suzqLcCwsL6du3LwBxcXGUlJS49m3evJn4+HhCQkKIiIggOjqa7du3s3r1aoKDg7n33nt57bXXXO8XotEVvAgto+HKO1QnER40MSWWsGAjB2hFodaVwcb1MmXz/3Bb7larFbPZ7HptNBqx2+2ufRERpx7hDg8Px2q1cvjwYY4ePcqcOXMYMGAAzz77rBeiC+HGnrXww1q48U9glAlQA8mw+CimW3oSFRnGEsd1dA/aw0uDImTK5l9wW+5ms5nq6mrXa6fTiclkOu2+6upqIiIiiIyMZMCAAQDcfPPN9c72hWg0BS9CWGuIH6U6ifCCYfFRFDw2gCcmTgIgmXWKE/kWt+WekJBAfn4+AMXFxcTExLj29erVi8LCQmpra6mqqmLXrl3ExMTQu3dvVq5cCcCGDRvo0qWLl+ILcQYHtuvLsV33RwgJV51GeFNkNHSIh22LVSfxKW5/Vk1OTqagoIC0tDQ0TWPatGlkZWURHR1NUlISGRkZpKeno2ka48aNIzQ0lDFjxvD444+TmpqKyWSSyzKi8a15GUxhermLwNfjdsibCpV79LIXGDTNN1aatVgs5OTkqI4hAsGRcnjxarjmD3DrDNVpRGM4uAteToCUaXDDQ6rTNKozdac8xCQCz7rXQXPKBGFNyUWdod1VsPVD1Ul8hpS7CCw1lbDxbX3oY6uOqtOIxtR9KPywDo7uU53EJ0i5i8Cy8S2wVclUA01Rj9v1j9s/VpvDR0i5i8BRdxzWvaFPM3BJL9VpRGNr2w0ujpFLMydIuYvAsXkBWPfLWXtT1n0olBVA9c+qkygn5S4Cg9OhD3/sEK9P7Suaph5D9ZvpcmlGyl0EiO2fwMFvZYKwpq59L2h1OWyVB5qk3IX/0zQoeAFaddJ/LBdNl8Gg/xn4biXUHFadRikpd+H/ygqgvBBufBiCjKrTCNV63K4vhF66RHUSpaTchd/KLSonMXM5y+f8jUO0ZLHhN6ojCV/QIQFaRDX5uWak3IVfyi0qZ3LOFsxHShlgLGZOXQqTPtwpa2gKCArSL818uwxqq1SnUUbKXfilGUtLqalz8CdTLtVaKHMdA2UNTXFKj6HgqIUdS1UnUUbKXfilvZU19DaU8lvjWuY4BnMUs2u7EFzWB8LbNulLM1Luwi9FtQxlavA77NVa87r91AgZWUNTAPqN9e63wc4vwHZMdRolpNyFX3q5+1Z6Bn1PZl06NTQDkDU0RX09boe6Y/BtnuokSki5C/9z/AjxO1/iYOsECiMGYACiIsOYbukpa2iKUzrepC+z2EQvzciqwcL/rHwOqn/morvep6BDvOo0wlcZTdDtVv1pVXstmEJVJ2pUcuYu/MvPO/WZH+NH6fPICNGQ7rdD7VHY/aXqJI1Oyl34l6V/heDmkPSk6iTCH1zRH0JbNsm5ZqTchf/Y8Tns/Bz6TwJzG9VphD8whULsLfoskY461WkaldtydzqdTJkyhdTUVDIyMigrK6u3f+HChVgsFkaOHMmKFSvq7Vu/fj39+/f3bGLRNNltsHQyXNQVrvuj6jTCn3QfCscr4ftVqpM0KrflnpeXh81mIzs7mwkTJpCZmenaV1FRwdy5c1mwYAFz5sxh1qxZ2Gw2APbt20dWVhZ2u9176UXTsX62PqXvLdPBFKI6jfAnXZIgOLzJXZpxW+6FhYX07dsXgLi4OEpKSlz7Nm/eTHx8PCEhIURERBAdHc327dupra3lySefZOrUqV4LLpoQ6wFY+Sx0HQRdk1WnEf4mOAxiBumXZpwO1Wkajdtyt1qtmM1m12uj0eg6G7darURERLj2hYeHY7Vaeeqpp7jnnnto166dFyKLJmfZU/rDKCnTVScR/qr7UKiugD1fqU7SaNyWu9lsprq62vXa6XRiMplOu6+6uprg4GA2btzIq6++SkZGBkeOHGHcuHFeiC6ahL1FUPQu9HkALu6iOo3wV10HgalZk7o047bcExISyM/PB6C4uJiYmBjXvl69elFYWEhtbS1VVVXs2rWLXr16sXTpUubOncvcuXNp2bIlzz//vPeOQAQuTYPPHoPwi6H/o6rTCH8WaobOSbDtI3A6VadpFG6fUE1OTqagoIC0tDQ0TWPatGlkZWURHR1NUlISGRkZpKeno2ka48aNIzS0aT0FJryoZBH8sBaGvgzNWqpOI/xdj9uh9BN91a7LrlWdxusMmqZpqkMAWCwWcnJyVMcQvsJWDS9fA+a2cP8KfQEGIS5ETSXM6ALXPwCDnladxmPO1J3yN0b4ptUvQNVeGPycFLvwjLBIuOI3sPVD/ZJfgJO/NcL3HC6DNS9BzxEQ3Ud1GhFIegyFyj2wb5PqJF4n5S58zxdPgCEIBv5ddRIRaGKHgMHYJKYBlnIXvuW7fP3H5pvGQ0uZm114WPhFcPlNTeLSjJS78B0OOyyZDJHRcOOfVKcRgarHUH0qiwPbVCfxKil34Tu+fhv2l+gjGYJlLVThJd1+CxgC/tKMlLvwDccOwfKn4fK++qPiQnhLRDuIvj7gn1aVche+4ctMOH4EbskEg0F1GhHoetwOB76Bg7tUJ/EaKXehTG5ROYmZy0mZ/Ab29f9md8dUaH+V6liiKej+W/3j1g/V5vAiKXehRG5ROZNztlBeeYwnTP/BqoVx164kcovKVUcTTUHLSyGqt5S7EJ42Y2kpNXUOBgVt5CbjN8yy/459dc2ZsbRUdTTRRJS0/A3sK+amyW+TmLk84E4spNyFEnsra7jUUMG04Dlsd17Ge46Bru1CeFtuUTnjt0QDkBK0nvLKGibnbAmogpdyF0p0aanxZvA/CcbOg3V/wYERgA6RMgRSeN+MpaXsqGvDN86ODDZuAKCmzhFQPzlKuYvG53QwN3I2XQzlPFj3F77TLgEgLNjIxJRYxeFEU3DyJ8SPHDdwTdAOehp219seCKTcReP7Ygrt96+k5OrHKWtxLQYgKjKM6ZaeDIuXKQeE9538CfFdx0AOaWYmmrLrbQ8EbhfrEMKjCt+Br16BPg8QN3gCBarziCZpYkosk3O2YK1rzqv2YTwR/C43s5XbU9JVR/MYOXMXjee7fPhkvL7c2aBnVKcRTdiw+CimW3oSFRnGu46B/MTFzLroQ4bFdVAdzWPkzF00joO7IDsDWneGEVlglD96Qq1h8VGnLgMWVcOHD+lrrPYIjOkv5MxdeF/NYZiXqs/Rnr5A1kMVvqdXGlwcC8v/oc9OGgCk3IV3Oerg/dFw+HtIfRdaX6E6kRC/ZjRB0hPw8w7YNF91Go9wW+5Op5MpU6aQmppKRkYGZWVl9fYvXLgQi8XCyJEjWbFiBQB79+5l9OjRZGRkMGrUKHbv3u2d9ML3LXkMdn8Jv30BLk9UnUaIM+t2mz4lwZfToe646jQXzG255+XlYbPZyM7OZsKECWRmZrr2VVRUMHfuXBYsWMCcOXOYNWsWNpuNF198kVGjRjF37lzGjBnDrFmzvHoQwketmw0b3oQb/wzxo1SnEaJhBgMMnApHy/U/t37O7V2twsJC+vbtC0BcXBwlJSWufZs3byY+Pp6QkBBCQkKIjo5m+/btTJo0iYiICAAcDgehoaFeii981rfLYMkkiL1V/wsjhD/o1A86D4BVMyHh99CshepE583tmbvVasVsNrteG41G7Ha7a9/JEgcIDw/HarXSunVrgoOD2b17N88++ywPPfSQF6ILn1VRql9nb9sDLLMhyKg6kRBnL2kK1BzSn8fwY27L3Ww2U11d7XrtdDoxmUyn3VddXe0q+7Vr1/LQQw/x3HPPccUVchOtyTh2COaNBFMo3DkfQiPcv0cIX9IhHq68A9a8AtYK1WnOm9tyT0hIID8/H4Di4mJiYmJc+3r16kVhYSG1tbVUVVWxa9cuYmJiWLt2Lc888wxvvvkmPXv29F564VvsNsgeBUf3Qdo8faFrIfzRzY+D/Tis+qfqJOfN7TX35ORkCgoKSEtLQ9M0pk2bRlZWFtHR0SQlJZGRkUF6ejqapjFu3DhCQ0OZNm0adXV1PPbYYwB06tSJp556yusHIxTSNPhkHJQVgOXfcNl1qhMJcf4u7gIJGbBhDlw/Flp1VJ3onBk0TdNUhwCwWCzk5OSojiHOQW5ROTOWlrK3sobx5s952P429JsIAx5XHU2IC3d0L7x04hLNHW+oTnNGZ+pOeYhJnJdTy+TVMCCokIfq3mGJ1ofcyLtVRxPCM1p0gD5jYNMC2L9VdZpzJuUuzsvJZfK6GfbwYvCrlGiX85faB5jx+U7V0YTwnMS/QGgLfVoCPyPlLs7L3spjDAtazcKQv2MljPttEzhOaEAtdiAEzVtD4p+h9FPYs051mnMi5S7OXc1hZjd/nRdCXmO7Fs3vbFPZT2sgsBY7EAKA6x+E8LaQN1UfOOAnpNzFufluFbx+E0naWp53ppFme4IftTaALJMnAlRIOPR/FPasgW/zVKc5a1Lu4uzYbfDFFHjntxDcjKD7PqfTHVO4JDJclskTgS/hbmh1OeT9HZxO1WnOiqyYINyrKIVF98JPW6D3HyDlGQgJZ1gUUuaiaTCF6A825dwH3+RAz9+pTuSWnLmLM9M0WP9v+Fc/fcxv2nx96t6QcNXJhGh8Vw2HdlfpI2fsNtVp3JJyF6dXtR/eGwGf/h9c3hce/Aq63ao6lRDqBAVB0pP6wjNF/1Gdxi0pd/FrpZ/B6zfA96vg1n/CXe9DRDvVqYRQr2syRN8IK58DW7X7z1dIyl2cYquGj/4C89P0p/P+uBKuu19fxEAIcWJBjyfBuh/W+e6UBCA3VJu0X84NM6BFOS8Ev0pEdRkkPqLfPDKFqI4ohO+Jvh5iBsPqF/UBBs1bq050WnLm3kSdnBtmX2U1Y425vFH7GFZrFatvfAuSn5JiF6IhSU+g1R7l3Znj6fTYJyRmLie3qFx1qnrkzL2JWvDZcu5zfsntIWvoErSXxY4beLzuD0R8HUnBINXphPBtuXsjMThv4nfax7xMEuWVrZmcswXwneHBUu5NSeUP+hjdkkUssG3CaTKwQYvledvv+MTZBzBQJXPDCOHWjKWlGOqGMzhkDeNNHzDJfj81dQ5mLC2VcheNpGo/bM2FkkXww4mJj6J685LpD8yzJvATF9X7dJkbRgj39lbWoNGWdx3J3GNaQu+gHcx3DGBRZT/V0Vyk3P3YL2+IdogMY2JKrH7WcOwQbPtIL/TvV4Hm1B++SJoCV1qgdSeii8o5krMF6hyurydzwwhxdjpEhlFeWcM0ezrfOC8n3bSMJ4Lf5dHgbMhZpt9ojb5e6UgzKXc/dfKGaM2Jcj5SeZA1/32N69duoX1FATjt0LqzvjLSlRZo263e+0/+6HjafxyEEA2amBJ74u8fLHL2Y5GtH1cH/8jznYu4ovQT2JwNbbrpJX91KoS1avSMUu4KnfHM2x27jfmfraC340eijQe4KWgLA4KKaGao46f9bSDxIf1R6fa9GjxzGBYfJWUuxHk43cnRH1Ju44r4MfrzIiWLYGMWLJkEeU/qJ1jX/AEuvbbRzubdlrvT6WTq1KmUlpYSEhLC008/TceOpxaLXbhwIQsWLMBkMvHggw9y8803c+jQIf7v//6P48eP07ZtW6ZPn05YmOev5Z53OfrA+//3zLu8subU3far2+tzuVSWweEy/WPlnlO/PrqXbDQ4MVrxgBbJfMcAPnLcQLHWhd3Jvz3rYxBCnJ8znhyFhEPC7/X/9m3SS37L+7BpHrS9Ui/5XiPJ3Wb16k/Obss9Ly8Pm81GdnY2xcXFZGZm8vrrrwNQUVHB3LlzWbRoEbW1taSnp5OYmMhrr73GbbfdhsViYfbs2WRnZzN69GiPhQY35XgWv0Fee7/mYNhVF4P9+C/+q/3Vxw2fbmSQw0qosY6LOcJlhgou4wCdFv8MHx3UL6u4GKBFlL4Ce6d+ENmRp9dUs9kayQ9aW36iFdqJRxai5IaoEL7jkqv1yfYG/QO2fACFWfDp/2Ff+gSOuuu5uO5myul8zv1zNtyWe2FhIX379gUgLi6OkpIS177NmzcTHx9PSEgIISEhREdHs337dgoLCxkzZgwA/fr1Y9asWR4v95NreD5szGGo8atTB7TYAAXuZy28+mA1iw0ahhB9ZRUD+sfgxcCqsBMrrmi/+Ei91zcePc6KICeGUA0jTkKpIxQbIYsdsNh9/mfAdeYNUKG14EetLYX2K4jqfxdERutlHtkRWl72q4eKroos571f/OMCckNUCJ8VGqGfsV/zByj/mk/feobBhlUMD13BSkcv7q6b5PGhlG7L3Wq1YjabXa+NRiN2ux2TyYTVaiUiIsK1Lzw8HKvVWm97eHg4VVVVHgn7SyfX6izXLqZUu9S13WCHTm0vcfv+rfv3cWrBLP0amAZodgPRHaJOXBcznPHjio0/6p+PATtGagnW/9NCGD+4J5iagSn0DB+b8fv/bOKHKie1WjCHMVNDM0A/8x46cIDb/HJDVAg/FZXAI9X3EE4aw4wFtOAYJzvIk2sQuy13s9lMdfWp2c+cTicmk+m0+6qrq4mIiHBtb9asGdXV1bRo0cJjgU86ORQpx9mPHOepsaVRkWEMGem+HKftXk75aX4joyLDGPo79+9/afuZ3z/+JvfvtwxuX++yDpz7mbfcEBXCP+n9Be86kn+13VPczi2TkJBAfn4+AMXFxcTExLj29erVi8LCQmpra6mqqmLXrl3ExMSQkJDAypUrAcjPz6d3794eC3zSxJRYwoKN9badSzmqfv+w+CimW3oSFRkmy9QJ0cRcaH+cDbdn7snJyRQUFJCWloamaUybNo2srCyio6NJSkoiIyOD9PR0NE1j3LhxhIaG8uCDDzJp0iQWLlxIq1atmDlzpscCn3ShlyVUv//k15AyF6LpaYzLqgZN0zT3n+Z9FouFnJwc1TGEEMKvnKk7ZcpfIYQIQFLuQggRgKTchRAiAEm5CyFEAJJyF0KIAOQzs0KWl5djsVhUxxBCCL9SXn76tVt9ZiikEEIIz5HLMkIIEYCk3IUQIgBJuQshRACSchdCiAAk5S6EEAFIyl0IIQKQ35S70+lkypQppKamkpGRQVlZWb39CxcuxGKxMHLkSFasWKEopWe5O+a3336bESNGMGLECF555RVFKT3L3TGf/Jz77ruP+fPnK0joee6OeeXKlYwcOZIRI0YwdepUAmH0srtjfuutt7BYLAwfPpwvvvhCUUrP27RpExkZGb/avnz5coYPH05qaioLFy70zDfT/MTSpUu1SZMmaZqmaUVFRdoDDzzg2nfgwAHttttu02pra7WjR4+6fu3vGjrmPXv2aHfccYdmt9s1p9Oppaamatu2bVMV1WMaOuaTZs6cqY0YMUKbN29eY8fzioaOuaqqShsyZIh28OBBTdM0bfbs2a5f+7OGjvnIkSNa//79tdraWq2yslL7zW9+oyqmR82ePVu77bbbtBEjRtTbbrPZtIEDB2qVlZVabW2tZrFYtIqKigv+fn5z5n62C3VHRES4Fur2dw0dc/v27XnzzTcxGo0YDAbsdjuhoaGqonpMQ8cMsGTJEgwGg+tzAkFDx1xUVERMTAzPPvss6enpXHzxxbRu3VpVVI9p6JjDwsLo0KEDNTU11NTUYDAYVMX0qOjoaF5++eVfbd+1axfR0dG0bNmSkJAQevfuzYYNGy74+/nM9APunM9C3f6uoWMODg6mdevWaJrGc889R48ePejUqZPCtJ7R0DHv2LGDjz/+mJdeeolXX31VYUrPauiYDx8+zLp168jNzaV58+bcddddxMXF+f3/64aOGeCSSy5hyJAhOBwOxowZoyqmR6WkpPDjjz/+aru3+stvyv18Fur2dw0dM0BtbS1//etfCQ8P58knn1QR0eMaOubc3Fz279/P3XffTXl5OcHBwURFRdGvX78zfTm/0NAxR0ZG0rNnT9q0aQPANddcw7Zt2/y+3Bs65vz8fA4cOMCyZcsAuPfee0lISKBXr15Ksnqbt/rLby7LnM9C3f6uoWPWNI2xY8cSGxvLU089hdFoPNOX8SsNHfOjjz7K+++/z9y5c7njjjsYPXq03xc7NHzMV155JTt27ODQoUPY7XY2bdpEly5dVEX1mIaOuWXLljRr1oyQkBBCQ0OJiIjg6NGjqqJ6XefOnSkrK6OyshKbzcbGjRuJj4+/4K/rN2fu57NQt79r6JidTifr16/HZrOxatUqAMaPH++RPxQqufv/HIjcHfOECRO47777ALjlllsC4sTF3TGvWbOGkSNHEhQUREJCAomJiaoje9xHH33EsWPHSE1N5bHHHuPee+9F0zSGDx9Ou3btLvjry6yQQggRgPzmsowQQoizJ+UuhBABSMpdCCECkJS7EEIEICl3IYQIQFLuQggRgKTchRAiAP0/yNwykRUCV1wAAAAASUVORK5CYII=","text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":"p_grid = np.linspace(0,1,20)       # 取20个离散的参数值\nprior = np.repeat(1,20)            # 20个参数值的不标准的先验分布\nlikelihood = binom.pmf(6,9,p_grid) # 投掷9次，6次海面朝上的各参数的似然\nunstd_posterior = likelihood*prior # 没有归一化的后验分布\nposterior = unstd_posterior/np.sum(unstd_posterior) # 后验分布归一化\nplt.plot(p_grid,posterior,'o')     # 画出20个值的点\nplt.plot(p_grid,posterior)         # 将20个值的点进行连线"},{"cell_type":"code","execution_count":5,"metadata":{"collapsed":false,"id":"B0DE008E6642497D86F5FBFD2376EFDD","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"data":{"text/plain":["[<matplotlib.lines.Line2D at 0x26b19a5d2b0>]"]},"execution_count":5,"metadata":{},"output_type":"execute_result"},{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAYIAAAD3CAYAAAAQYlNPAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA6uElEQVR4nO3deXwU9fnA8c/MbDaEHAiCgmjUYqOCxCRIQSmIQsQKWgxNwECK4oHYVjm0eFCwlEsrtL9KaeuFGkQSKsUb5JIgp0YiokY0KiAqRAOSLHE3OzO/P8JMdjgSjg2b3X3er1dfknm+u8yUZJ98n++lmKZpIoQQImqpob4BIYQQoSWJQAghopwkAiGEiHKSCIQQIspJIhBCiCjnCvUNnIhu3brRvn37UN+GEEKEjV27drFx48YjxsIyEbRv355FixaF+jaEECJsZGVlHTUmpSEhhIhykgiEECLKSSIQQogoJ4lACCGinCQCIYSIcmE5a0gIEVyLN+9ibEEJRj1thnVPZsrAzqfsnsSpI4lAiCiWOettPtvjOaa28zbsYN6GHYAkhUgjiUCIKLN48y7uW1hCzSG//rdiP2nq55yjlNMCD15i+M5sySfmuWwzzwYUu+28DTuYv3EHs3LSGJguizvDnSQCIaLI0CfXs7aswv66GV4GaWu4UXuHy9RtR33dN2YrXtUv5zl/P76hNQCGCaMLSvjnqs9YNrZ3Y9+6aESSCISIEt2mLmN3pQ8ADZ08bRm/c71MG+VHPjXO5q81OWwyLqLMPIsficdNDWcr35Omfk6mWsyt2pvcqr1Jvp7J3/yD2E8CAJ/t8XDRQ29QOvW6UD6eOAmSCISIcIs372J0QYn99UXKDh6N+Q+p6pes0zsyyn8P75kXElj6AahG4zPzbD7Tz2ah3puz+J67XC/zW+0trtM2Mrrmd6w3OgHwk25KMghjMn1UiAh2aBIYpBbxsvtPtFN+4C7f3eTWPMR75kUcmgSO5BtaM8F/K7/2/YUqM44XYqZxl/YyUHva7U+6SeqkJY3zIKJRSSIQIoKNOZgEVAwmup5npvvfFBs/p5/3Ud4wunO0BDCsezLDuicfMbbV/BnX+6bymtGdP8YUMNn1LOrBiaf7vbokgzAkpSEhIlTqpCWY1I4HPBbzb27U1jLX348p/mHoaIe179GhFS/cfrnjmjVFdMLiD+2powAHaMY9Nb/jG/N07nS9Rhxe/ui/AxOV/V6dblOXsfGhzEZ9PhE80iMQIgKlTlrCfq+OCz+PxzzOjdpaHq3J4c/+4YclgZ+fEc9XM/oflgQCTRnYma9m9Hf0EkxUZvhz+bs/i2xXERNd+Vhlot2VPjJnvd0YjyYagSQCISKMlQTA5JGYJ7lO28Tkmjzm6AMPazuse/JxTf20EkIzra6k9Hf/IJ72/4pbXEu5VXvDvv7ZHg9Dn1x/Ek8iThVJBEJEkG5Tlx1MAnC/awGDtDXMrPkNz+i/crRLitX4akb/E14dXDr1OpJirZ6FwhT/UN7Qf8GDrvn0VLfY7daWVTBh8Ycn9HeIU0cSgRARYuiT6+11AkO0ldzpepXn/Zk8rt/oaJcUq7Hlz9ee9N+35c/X2kPNJir31tzJNvNsZsf8g7P43m43b8MOFm/eddJ/n2g8kgiEiACLN++yVwxnKNuY7JrLaj2Vh/3DCZwZFKwkYPnb4DT7zwdoxsiasaiY/M09x55JBHDfwpKg/Z0i+CQRCBEBxhaWANCGvfzL/Xe+NU/n7prfYwT8iAc7CQAMTG/vGEDeYZ7JpJrhdFNLuVN71b5eYyC9giZMEoEQYS5z1tsYZu1agcfds0mkmpE1Y/nx4BYQlmAnAcuUgZ0dyWCR0ZNX9e6Mcf2XTspX9vWxAQvbRNMiiUCIMLZ48y57G+mR2mt0Vz9hov9mSk3nYrC/B5RwGsOUgZ3p0aHVwa8UHqoZwT4SmBbzlF0iMkBmETVRkgiECGNWSaiz8gVjXQt5Te/Of/VejjbDuiefkq2iX7j9clxq7XjEfhKYXJPHpeoX/FZ7y26ztqxCSkRNUIOJwDAMJk6cyODBg8nLy2P79u2OeGFhIVlZWeTk5LBq1SoAKioqGDFiBLm5uYwePZrq6moAnn32WbKzs8nOzmb27NkAmKZJz549ycvLIy8vj5kzZwb7GYWISN2mLsMwIY6f+HvMPymnBQ/WjCBwcLhHh1an9ACZx7Ivtf/8qnE5q/VU7nUV0pYf7OtSImp6GkwEy5cvx+fzUVBQwLhx45gxY4YdKy8vJz8/nwULFvD0008za9YsfD4fc+bMYcCAAcyfP5+OHTtSUFDAzp07eeWVV1iwYAGFhYW88847lJaWsmPHDjp16kR+fj75+fmMGzeuUR9YiEgQOFV0jOslOqjfMq5mlL01tKW+1cKNYWB6e0eJaIL/FmLQ+WNMgd3GAFlb0MQ0mAiKi4vp2bMnAGlpaWzdutWObdmyhfT0dNxuN4mJiSQnJ1NaWup4Ta9evVi3bh1t27blqaeeQtM0FEXB7/cTGxvLRx99xO7du8nLy+P222/niy++aKRHFSIyBE4VTVXKuFV7gxf8fewtoS2NPS5wNIElop3mmTyl/4os7R1SlTK7TeC+RSL0GkwEVVVVJCTU/ZahaRp+v9+OJSYm2rH4+Hiqqqoc1+Pj46msrCQmJoZWrVphmiaPPPIIHTt25Pzzz6dNmzbccccd5OfnM3LkSO67775gP6MQEWX8S7Urd134eSTmSb6nBTP8Nzna9OjQKqRHSAaWiP7lv4FyM4kJMfOw9iICmU7alDSYCBISEvB46g63NgwDl8t1xJjH4yExMdFx3ePxkJSUBIDX6+Xee+/F4/EwadIkAC655BL69OkDwGWXXcaePXswzbpvFiFEncWbd+H1187CuUN7jYvVHfyp5hYqaW63OTPRfcpLQocKLBFV0ZxZ/mx+oX5KP/Vdu42MFTQdDSaCjIwMioqKACgpKSElJcWOpaamUlxcjNfrpbKykrKyMlJSUsjIyGD16tUAFBUV0aVLF0zT5K677uLCCy9k8uTJaFrtPiWzZ8/mueeeA6C0tJR27dqhKA0fkiFENLp34QcAnK2Uc7frf7yh/4K3jK6ONk1l++fAZFSo9+Zz4yzGuRY6ppPKWEHT0OB5BJmZmaxdu5YhQ4ZgmibTpk1j7ty5JCcn06dPH/Ly8sjNzcU0TcaMGUNsbCyjRo1i/PjxFBYW0rJlS2bOnMny5cvZtGkTPp+PNWvWADB27FjuuOMO7rvvPlavXo2maUyfPr3RH1qIcDRh8Yf4jdre8oOuFzBQ+UtNnqNNqMYFjmZY92TmbdiBjsbf/YOY7X6c/uoGXjWuAGrHCk7lrCZxZIoZhnWYrKwsFi1aFOrbEOKUOu/+1wG4Qt3KfPc0HqvJZnbAhnIxKnw2rX+obu+orPtWMFjivh8Ng2t8j9rbXwzrnizJ4BSo73NTFpQJEQasEoqGziTX8+ww2vCk7vzQ/2t2WgjurGHW9hMmKn/z/4YL1G+4Xl1nx2UGUehJIhAiDFgflsO05Vyofs0U/zC8uO14qGcJ1WfKwM72dNKlxmV8bJzLPa5Fjt1JZQZRaEkiEKKJs/bnaUEVY10LWaNfwlvGZXZc5dQvHDte1nRSE5XH/QP5mfod16jv2XHZpjq0JBEI0YQFLh4b5XqFRKqZ6h9G4DYSs5rYAPGRBPZWlhpd+dI4kztdr2CtK5BtqkNLEoEQTZi1eKwdP3CLtpT/Gb88bGfRploSOpQ1VmCg8qQ+gDT1C7oppXZcegWhI4lAiCYqcPHYGNd/AZNZNb9xtAk8B6CpC5wZ9JLek3IziZEuObymKZBEIEQTZS0eS1F2Mkgr4nn9GnbRxo6rEHbTLq3E5cXNs/5ruVor4UKlbtaQ9ApCQxKBEE1Q4OKx+1wFeGjGP/2/drQJh7GBQwUmrnl6X6pNN8O1pfY16RWEhiQCIZqgFw5OF+2ifEqm9j7/9t/APuo2eGzK00UbYvUKfiSB/+k9uFFbSxJVdvzhVz4K1a1FLUkEQjQxizfvsvfoHOP6L+VmEnP1fo42TX26aH0CewX5+jXEKT6ytdX2tX3VNaG4ragmiUCIJsaaKdRVKeWX2kf823891TSz4+E0QHw01jN8Yp7LRuMifqstQ5EFZiEjiUCIJiRwptA9rpcoN1vwgt7X0SbcBoiPJPAZnvdfw7nqHnqrH9jXHli0JRS3FbUkEQjRhFgzhQJ7Az8Ra8dPi4sJ1a0F3cFdJ1hqXMZu8zSGBxxyX11jSK/gFJJEIEQTEThTaLTdG+jjaPPwDZ2O9NKwlNuttjzkx8UL/r701j4gWdltx2XQ+NSRRCBEE2FtLPcL5RN6HKE3EM4zhY4ksDxUqF+JbioyaBwikgiEaAICT+q6x7WIPeZpzDtkbCCcZwodTcvmtaWu7zid1calZGur0dDtuJxgdmpIIhCiCbDWDWQo2+ihfcR//P0d20xHwkyhI5l0fV2pq0C/irbKXnqpdQPFL8hZBaeEJAIhQixw3cAo16vsNRN4MWBsIBy3kjhWA9PbE++uPb98hZFOuZnEEG2VHQ+74xPDlCQCIULsof/Vlj9SlJ1kasU86+/HgYB1A+G4lcTxmHpjbZLz4+IlvRd91Pdpwz47LrOHGp8kAiFCaPHmXXh8tTXxO12v4jFjeU6/xtEmkgaIjyTw+RbqV+JSDLK0NfY12Yiu8UkiECKErN7A2Uo5N6jreFG/2rGnUCStG6iPNWhcZrZnk3EhOdrbyKE1p44kAiFCJLA3cLv2GgYKT/mvc7SJpHUD9QkcNC7Ue9NB/ZYuyjb7mqwpaFySCIQIEas30JofGay9zSK9J99xuh2Pi1EjvixkGZje3j588039F1SbbgZqa+24rCloXJIIhAgRqzdws2sJbvw8oQ9wxKdnpYbitkJm6MEpsh7ieMu4jAHaBmLw23EpDzUeSQRChIC1UCqBA/xWW8abRle+MM+y49HUG7AETpH9n96DlkoVvdUS+5oMGjceSQRChIC1UCpHW02ScoD/+K93xKOtN2CxBo3fMTrzvZnEQO0dOyaDxo1HEoEQp5i1gEzF4GZtCe8aKWwxO9jxGDXyp4wejTVo7MfFq/rl9FU3k4THjsugceOQRCDEKWYdPNNHfZ9ktZy5/msd8b9mp4XgrpqGwEHj/+m/JFap4Vptkx2XQePG0WAiMAyDiRMnMnjwYPLy8ti+fbsjXlhYSFZWFjk5OaxaVbs0vKKighEjRpCbm8vo0aOprq4G4NlnnyU7O5vs7Gxmz54NwE8//cQf/vAHcnNzuf3226moqAj2MwrRZAQePHOLtoRd5uksNbra8WjuDVisQeMt5s8oM9pxo7q2gVeIk9VgIli+fDk+n4+CggLGjRvHjBkz7Fh5eTn5+fksWLCAp59+mlmzZuHz+ZgzZw4DBgxg/vz5dOzYkYKCAnbu3Mkrr7zCggULKCws5J133qG0tJQXX3yRlJQU5s+fz8CBA5kzZ06jPrAQoWT1Bi5SdnCF9jHP+69BR7Pj0dwbsNQNGiss1ntwufYxZ/G9HZdxguBrMBEUFxfTs2dPANLS0ti6dasd27JlC+np6bjdbhITE0lOTqa0tNTxml69erFu3Tratm3LU089haZpKIqC3+8nNjb2sLbr169vjOcUIuQO7Q1Um24W6Fc52kR7b+BQi40eANygrbOvyeyh4GswEVRVVZGQkGB/rWkafr/fjiUm1i2Hj4+Pp6qqynE9Pj6eyspKYmJiaNWqFaZp8sgjj9CxY0fOP//8I7YVIhJZC8hasZ+B2loW6T35kbqfrUjdavpEWLOHdppnstm4gP7aBjsms4eCr8FEkJCQgMdTN2pvGAYul+uIMY/HQ2JiouO6x+MhKSkJAK/Xy7333ovH42HSpEmHvUdgWyEijbWA7CZtJbFKDXP1fo54pG41fSICt5x4Te9GZ/UrzlW+s6/J7KHgajARZGRkUFRUBEBJSQkpKSl2LDU1leLiYrxeL5WVlZSVlZGSkkJGRgarV9ceOVdUVESXLl0wTZO77rqLCy+8kMmTJ6Npmv3+h7YVItJYC8hc+MlzLaNI78zn5tl2XHoDTs4tJ7oB0F+t6xXI7KHgcjXUIDMzk7Vr1zJkyBBM02TatGnMnTuX5ORk+vTpQ15eHrm5uZimyZgxY4iNjWXUqFGMHz+ewsJCWrZsycyZM1m+fDmbNm3C5/OxZk3tFrNjx47lpptuYvz48dx0003ExMQwc+bMRn9oIU416zzi69RNtFX2cr9+myMuvYHDDe2ezLwNO/iG1hQbP2eAtpE5+kA7vnjzLhlTCRLFNM2wOwQoKyuLRYsWhfo2hDgmExZ/aCeCxe4/kcgB+vr+inmwQx4Xo/LJX34Vyltsss67/3UARmhvMjEmn6u9j9lbcZwWF0PJpGvqe7kIUN/npiwoE6KRWdtJpCufkaaW8Zx+jZ0EIHq3kzgW1qDx6wfLQ9epG+2YlIeCRxKBEI0o8DziW1xL2G8257/6lXZcFpDVzxo03k0rNhkXOmYPieCRRCBEI7KmjLblB36lbqJA7+04j1gWkNUvMEm+rnfnYnUnHZS6qaMyjTQ4JBEI0YisKaN5rmWoGI7ziKU3cHze1H+BYSr0DygPPbBoSwjvKHJIIhCikVhTRmPxcZO2kuVGF742z7Dj0hs4NtY4wR5a8q7pLA9V1xjSKwgCSQRCNBJrptBAbS2tlCrm6s5dRqU3cGyci8u6c6H6NRcoX9vXZHHZyZNEIEQjsHoDYHKLtoSPjXPZYFxsx+Ni5EfvWAUmzKV67U6t/dT37Gsye+jkyXejEI3AmjJ6ufoxF6k7D24nodhxmTJ6fALLQ+8bF9BPe9cRl/LQyZFEIESQBU4ZHaG9yfdmEq/oV9hxGSQ+foHloaV6V1LVL2lPuX1NykMnRxKBEEFmnTmQrOymj7qZ+frVeHHbcRkkPn4D09sT767dn2ypcRkA12hSHgoWSQRCBFHgmQM3a0vxozLPn2nHpTdw4qbeWLsf01dmO0qNc+gXkAhAykMnQxKBEEFkLSBL4ADZ2mpeN7qzh5Z2XHoDJ84xaGxcRlellFbst69JeejESSIQIoisBWTZ2moSlerDDqaX3sDJsQaN39K7oikmfbViOybloRMniUCIILFKEyoGw7W3KDZ+zhazgx2XMwdOnjVo/JF5LjuNNo5ppOLESSIQIkisQeKr1M2cp+4+rDcgZw6cvLoelcJS4zJ+qX5IPNV2XMYJTowkAiGC4NCD6b8xW7HE6GrHZQFZ8C3VuxKr+OmtfmBfk3GCEyPfnUIEgdUbSFF28kvtI/L91+APOABQFpAFjzVOUGymUG4mORaXyTjBiZFEIMRJOrQ3UG26eVG/ytFGBomDxxonMFBZpnfhKrUEN3UJQMpDx08SgRAn6c+v1pYjWrKfG7V3+J/eg30k2nEZJA6uwKT6ltGVRKWaK9St9jUpDx0/SQRCnKS9B2p/G71JW0UzpYa5uvP8YRkkDj6rPLTO6ESlGUem+r4dk/LQ8ZNEIMRJsMoQLvzkuZaxRr+Ez8yz7bj0BhqHVR7yEcNqI5U+2vsoGHZcykPHRxKBECfBGiT+lbqJdkrFYWcOSG+gcQSWh1boGbRV9nKJ8pV9TcpDx0cSgRAnyDFI7FrCl8aZrDLS7LhylNeJ4LDKQ6uMNHRTkVXGJ0ESgRAnyNpX6FLlczLUz3lO74cZ8CM1VMpCjcoqD+0jkffMCx3jBOL4SCIQ4gQs3rzL3lfoFtcS9ptxLNSvdLSRslDjCiwPLdO70FHd7jijQMYJjp0kAiFOgNUbOIO99Fc3slDvjYc4Oy6DxKfWCiMDgKu1zfY1GSc4dpIIhDhOgb2BPNcyNAye1a9xtJHewKlhjRN8abajzGhHpirjBCdCEoEQx8laQBaLj1xtBSuMDHaaZ9rx0+JiQnVrUSfwCMtlRhe6qx+TwAH7mpSHjk2DicAwDCZOnMjgwYPJy8tj+/btjnhhYSFZWVnk5OSwatUqACoqKhgxYgS5ubmMHj2a6uq63QErKiro168fXq8XANM06dmzJ3l5eeTl5TFz5sxgPp8QQWctILtBW8fpSiXPHDJl9OEbOh3pZaIRBI4TLNczcCs6vdQt9jUpDx0bV0MNli9fjs/no6CggJKSEmbMmMG//vUvAMrLy8nPz+ell17C6/WSm5tLjx49mDNnDgMGDCArK4snnniCgoICbr75ZtasWcPMmTMpL68b0NmxYwedOnXi3//+d+M9pRBBUvcbpskIbQmfGOew3uhox+NiVNlX6BRr2TyGvQdqeN9MocJMoK/2Pm8Y3QEpDx2rBnsExcXF9OzZE4C0tDS2bq3b02PLli2kp6fjdrtJTEwkOTmZ0tJSx2t69erFunXrav8yVWXu3Lmcdtpp9nt89NFH7N69m7y8PG6//Xa++OKLYD6fEEFlDRJfrn7MxeqOgwvI6lYMyC6jp17gJnSrjHSuVjejodtxKQ81rMFEUFVVRUJCgv21pmn4/X47lphYt7lWfHw8VVVVjuvx8fFUVlYC0KNHD1q2bEmgNm3acMcdd5Cfn8/IkSO57777Tv6phGgEjimj2hIqzARe1ns42khv4NQ7dBrpaYqHLso2+5qUhxrWYGkoISEBj8djf20YBi6X64gxj8dDYmKifb1Zs2Z4PB6SkpKO+v6XXHIJmqYBcNlll7Fnzx5M00RRZF2maFqs3sA5ym76qu8zR78BL247LlNGQ8cqD60xOuM1XfTV3meT/2JAykPHosEeQUZGBkVFRQCUlJSQkpJix1JTUykuLsbr9VJZWUlZWRkpKSlkZGSwevVqAIqKiujSpctR33/27Nk899xzAJSWltKuXTtJAqLJCewNDNfeQkcl35/paCNTRkPHKg95iGOD0ZFM9T3AtONSHqpfg4kgMzMTt9vNkCFDmD59Og888ABz585lxYoVtGnThry8PHJzcxk+fDhjxowhNjaWUaNG8frrrzNkyBA2b97MsGHDjvr+d9xxB++++y7Dhg1j+vTpTJ8+PagPKEQwWFNGEzjAYO1t3jC6sZtWdlymjIaWozxkdOF8dTcdlG/sa1Ieql+DpSFVVZk8ebLjWocOHew/5+TkkJOT44i3bt2ap59++qjvuXLlSvvPLVq04IknnjjmGxYiFKwpo9naahKVap7xy5TRpsYqD63QM5gSM5e+6vuU6bUJQspD9ZMFZUI0wCorqBjcrC3lPSOFD8wL7LhMGW0arPLQt5zOVuM8x26kon6SCIRogDVI3Ed9n3PVPYf1BmTKaNPgOKPAyCBD+YxW7LevyTjB0UkiEKIegYPEt7re5GuzNUuNro420htoepbpGWiKyVVqiX1NxgmOThKBEPWwegOdlK/orn7Cc/5r0NHsuAwSNy3WJnRbzfP5zmwph9UcI0kEQhxFYG9ghOtNPGYsBfpVjjYySNy01G1Cp7BCz6CXuoVYfHZcykNHJolAiKOwegNt2Mf16joW6leyn3g7LoPETY9zGmkG8YqX7uon9jUpDx2ZJAIhjiCwNzDMtRwXBs/q/RxtZJC4abLKQ+uNThwwY+kTcISllIeOTBKBEEcQeObAUG05K4x0vjLb2XHpDTRdVnnIi5s1Rmf6aO8jq4zrJ4lAiCMIPHOgtbKfZ/RfOeLSG2i6HGcUGBm0V36go1J3joqUhw4niUCIQzjPHHiTT4xkOXMgzFjloVV6OoapSHmoAZIIhDiENUh8hfoRF6s7D55AJmcOhBOrPPQ9LSgxO9BXe7+BV0Q3SQRCBHAsINPepNxM4hX9Ckcb6Q00fc4jLLtwqfoFZ7DXvibjBE6SCIQIYPUGLlC+po+2mXn+TMeZA7KALPwsNzIADg4a15JxAidJBEIcFNgbuF17g2rTzfO688wBWUAWPqxxgm3m2eww2sg4QT0kEQhxkDVl9Az2cqO2hkL9SvZSd7qeDBKHF8cqYyODX6pbaYbXjkt5qI4kAiEOsqaM3uJagobBU/p1jrgMEoeXQw+raabU8Et1q31NykN1JBEIQd1vh/FUM1RbwZvGL9hpnmnHpTcQnqzy0LvGRew34+iryiZ0RyKJQAhg/EtbABiirSRJOcAT/gGOuPQGwpNVHqrBxWrjUvpom1Ew7LiUh2pJIhBRb/HmXXj9Bi783Op6kw3GxWwxOzjaSG8gPDmnkWbQRvmRS5Uv7GtSHqoliUBEPas3cL26nrOUCv5zSG9gWPfkUNyWCBKrPPS2kYbfVB3TSKU8VEsSgYhqVm8ATO5wvcanxtm8bVzqaDNlYOfQ3JwICqs89CMJvGde6BgnACkPgSQCEeWs3kAvdQsXqzt5Uu+PGfBjIb2B8OeYPaRncLG6k7OVcvualIckEYgoVtcbgDu01/jObMnLeg9HG+kNRAarPLTCWmUsi8scJBGIqGX1BlKVMn6pfcRc/7XU4LLj0huIHFZ56CuzHZ8bZzkSgZBEIKJUYG/g967F7DPjmaf3dbSR3kDkOPSMgu7qxyRwwL4W7eMEkghEVLI2l7tQ2cE1WjHP6v3wEGfHpTcQuZbrGbgVnV7qFvtatI8TSCIQUSdwc7nfuV6mymzGXP+1jjbSG4g81jjB+2YKFWaCTCMNIIlARB2rN3Ce8i391Q3M0zP5kQQ7HhcjPxaRyBonMFBZZaRztVqChm7Ho7k81OB3vGEYTJw4kcGDB5OXl8f27dsd8cLCQrKyssjJyWHVqlUAVFRUMGLECHJzcxk9ejTV1dV2+4qKCvr164fXW7sL4E8//cQf/vAHcnNzuf3226moqAjm8wnhENgbGKW9Sg0unvLL5nLR4NBVxi2VKjKUz+xr0VweajARLF++HJ/PR0FBAePGjWPGjBl2rLy8nPz8fBYsWMDTTz/NrFmz8Pl8zJkzhwEDBjB//nw6duxIQUEBAGvWrGHEiBGUl9fN4X3xxRdJSUlh/vz5DBw4kDlz5jTCYwpRy5op1J5ysrQ1LNCv4nta2HHZXC6yWeWhIiMVr+miryab0MExJILi4mJ69uwJQFpaGlu31m3jumXLFtLT03G73SQmJpKcnExpaanjNb169WLdunW1f5mqMnfuXE477bQjvn+vXr1Yv3590B5OiECOdQOu1zBBNpeLMlZ5yEMcG42L6XvINNJoLQ81mAiqqqpISKirn2qaht/vt2OJiYl2LD4+nqqqKsf1+Ph4KisrAejRowctW7Y87P2P1FaIYLPGBtqwjyHa2yzSe/INre249AYi36FnFHRQv+V85Vv7WrSWhxpMBAkJCXg8HvtrwzBwuVxHjHk8HhITEx3XPR4PSUlJHM3xtBXiZFhjA7e5XseFn3/pNzji0huIDlZ5aKWeDsgqYziGRJCRkUFRUREAJSUlpKSk2LHU1FSKi4vxer1UVlZSVlZGSkoKGRkZrF69GoCioiK6dOlS7/sfa1shTtTQJ2tLjm3Yx2+1Zbxs9GC72daO9+jQSnoDUcIqD+2iDR8b55KpySZ0DSaCzMxM3G43Q4YMYfr06TzwwAPMnTuXFStW0KZNG/Ly8sjNzWX48OGMGTOG2NhYRo0axeuvv86QIUPYvHkzw4YNO+r733TTTXz22WfcdNNNFBQU8Pvf/z6oDyjE4s27WFtWOxttlOsVYvDzf/4sR5sXbr88FLcmQsC5yjidLso2TqOuJB2N5SFXQw1UVWXy5MmOax061B3akZOTQ05OjiPeunVrnn766aO+58qVK+0/x8XF8Y9//OOYb1iI42WNDbTlB4ZqK3hJ7+XoDci6gejTsnkMew/UsFzvwt2uxfRWP2Cx8UsgOstD8hMgIlrguoG7XK+gYvC4fqOjjYwNRB+rPPSheT67zdO4RnsvxHcUWpIIRESzegPtKWeItpJCvTdfm23seIwqx1BGI+vf3ERlqd6V3uoHNMNrx6NtnEASgYhoVm/g967FmCjM9g90xP+anXbqb0o0KUuMrjRXvFG9CZ0kAhGxrJlCycpusrXVvKhfzbecbsdl3UB0s6aRbjQuZq+ZwLXau3Ys2sYJJBGIiBQ4U+hu1//wozHH/2tHGxkbiG7WOIGOxjK9C33V94nBb8ejqTwkiUBEJGtPoRRlJzeqa8jXM9lD3ap2GRsQgf/+S4yuJCkHuEKtKwlFU3lIEoGIOIF7Co13LcBDHP88pDcgYwMC6spDa41LqDTj6KdusmPRVB6SRCAizr0LPwCgm/IJfbTNzPHfwD7q9sSS3oCwWOUhL25WGWlcoxWjYtjxaCkPSSIQEWXC4g/xGyZg8kDMfL4xWzFXd54+Jr0BYQn8heBN/Re0VvbTVfnUvhYt5SFJBCKizNuwA4Dr1I2kqWX8zf8bvLjtuOwpJA5llYdWG5fykxnDtVr0lYckEYiIYU0XdeHnPlcBnxpn85Ley9FG9hQSh7LKQwdoRpGRSj/tXZSA8lA0kEQgIkLgdNEh2irOV3fziH8IRsC3+LDuyaG6PdGEHVoeOkupIFX5wr4WDeMEkghERLC2kkjCw2jXS2w0LmKlke5oM2Vg51DcmggjK4x0akyNXwUsLrtvYUnobugUkUQgwl7gxnL3uBbRikom1/wWUOw20hsQ9bHGCfaTwDqjE79SNwImADVG5PcKJBGIsGctHuug7OK32lss0HvzkXmeHVeR3oConzVOAPCa0Z1z1T10Vr60r0X67CFJBCKs1S0eM5noyqeaWB7zD3a0mTU4LST3JsLHwPT2dv9xqd4Vn6lxvbbejkf67CFJBCKsWYvHrlY3c6W2hb/7B1FB3bnXsrGcOFZDD5YP9xPPaiONAdp6x+yhSC4PSSIQYWvok+vxGyZuaviTK5/PjbN4Xs90tJGN5cSxCiwfvqpfzllKBV2Ubfa1SC4PSSIQYSlwuuht2hucr+5msj8Pf8Dpq7KVhDhe1qDxciODatPNDVFSHpJEIMKSVRI6R9nN3a5FLNG7UmRc6mgjW0mI4xW4uGyFkcF12kY0dDseqeUhSQQi7ATuJzTFNZcaXEyqGe5oI1tJiBMR+D3zqn45rZX9XK5+bF97YNGWI70s7EkiEGHH2k9ogLqBK7UtzPRns5tWdlxFtpIQJ84qD71tXEqlGcf1al15qLomMreekEQgwkrmrLeB2hXEk2KeZ4txPs/r1zjayHRRcTICt6Z+y7iMa7VNuKkbH4jE8pAkAhE2hj65ns/2eAD4o2sBrdjPAzW3OfYTkpKQOFmB3z+v6FfQQjnAVWqJfS0SZw9JIhBhIXCW0C+UT8jVVvKsfi0fmec72klJSARDvFsD4B3jEvaYpzFIK7JjkTh7SBKBCAvWNhJx/MRfY/7DTrMNj/mzHW1kPyERLFNvrF1ToKPxP70HV6kltGK/HZ+w+MNQ3VqjkEQgmrxDzyA+Rynn3po7qaaZ3Ub2ExLBFFgeeknvRYyic4O2zr5mTViIFJIIRJM3trAEgMvVj7jZ9RZz9Wt517zI0UYGiEWwWbOHtpnn8KFxHr8JKA9BZA0aN5gIDMNg4sSJDB48mLy8PLZv3+6IFxYWkpWVRU5ODqtWrQKgoqKCESNGkJuby+jRo6murj5q23379tGtWzfy8vLIy8vjueeeC/YzijDWbeoyDBPiqeavMf/hC6Mtf/XnONoM654sA8Qi6AJ3JH1J78Ul6ldcqNT1BCJp0LjBRLB8+XJ8Ph8FBQWMGzeOGTNm2LHy8nLy8/NZsGABTz/9NLNmzcLn8zFnzhwGDBjA/Pnz6dixIwUFBUdt+/HHHzNgwADy8/PJz89n+PDh9dyNiCaZs95md6UPgEmu5zmLH7i35k5+ItZuE6NKSUg0joHp7Yl11X5Evqxfgc/UGKStseORNGjcYCIoLi6mZ8+eAKSlpbF161Y7tmXLFtLT03G73SQmJpKcnExpaanjNb169WLdunVHbbt161Y++ugjhg0bxt13382ePXsa6VFFOJmw+EN7qugN6jpyXKuZrf+a980URzvZRkI0pkcG1W5auJckVhnp3Ki9E5FbTjSYCKqqqkhISLC/1jQNv99vxxITE+1YfHw8VVVVjuvx8fFUVlYete3PfvYz7r77bubNm0ffvn2ZMmVK0B5OhC9rMO4cZTdTY57mPSOF//MPcrSRNQOisTkHjXvSRvkxItcUNJgIEhIS8Hg89teGYeByuY4Y83g8JCYmOq57PB6SkpKO2rZ79+5069YNgMzMTD7+uG5fDxGdrNXDLvw8HjMbE4V7fL9DR7PbnJnoljUD4pRQD55Ys9JIZ7d5GrnaCjsWKeWhBhNBRkYGRUW1o+UlJSWkpNR1zVNTUykuLsbr9VJZWUlZWRkpKSlkZGSwevVqAIqKiujSpctR206YMIGlS5cCsH79ejp16nT4TYiokTnrbbskNM61kDS1jPtrbmMXbRztNj6UeaSXCxF0ud1q16f4cbFAv4re6ge0p9yOR8KaAsU0TbO+BoZh8PDDD7Nt2zZM02TatGkUFRWRnJxMnz59KCwspKCgANM0GTlyJP369eP7779n/PjxeDweWrZsycyZM2nevPkR2+7cuZMHH3wQgLi4OKZMmcIZZ5xR701nZWWxaNGi4P2/IJqECYs/tEtC/dR3+Y/7b8z3X82D/tsc7f4+OE1KQuKUOu/+1wE4i+9ZE3sPc/RfMzNg9tpXM/qH6taOWX2fmw0mgqZIEkFksn7YOii7eNn9Jz4325Pjm4iPGLtNjw6tpCQkTrn0yW+x90BtGejJmMdIU8u43Pu4fRDSsO7JTX72Wn2fm7KgTDQJqZOWAJDAAZ6ImUU1bu70jXYkAdleWoRK4JqC+Xof2ig/kqkW29fCfaWxJAIRcqmTlrDfq6NgMDPm35yr7Ob3vnv4jtMd7WT1sAiVwDUFq41L+dpszVBtuaNNOE8llUQgQqrb1GXs99bOy77PVUg/7T2m+Yey0bzY0U5WD4tQs9YUGKjM91/NL7WP+LnytR0P59PLJBGIkBn65Hp75fAQbSV3uV7hBX8fntGvdbTr0aFVk6+/isgX2Ct4Ub+aatPNCO1NOx7Op5dJIhAhMWHxh/b5Ar3UD5jieoa39UuZ6L8ZUOx2sl5ANCWBK41f0nuSpb1Da3604+E6lVQSgTjlhj653h5c66R8yT9j/sE28xx+V3O3Y9FYUqwm6wVEkxJYnnxav45YpYZhrmX2tXAdNJZEIE6poU+ut3sCFyhfk++ezo/EM8J3Lx7i7HZJsRpb/nzt0d5GiJCxTi/70mzHMj2DYdpyYvHZ8XDsFUgiEKdMYDnoHGU389zT8eNiqO/Bw2YISRIQTZV1ehnU9gpaK/sdZxWEY69AEoE4JQLLQe34gRdiphFLDcN8D7DdbOtoK0dOiqYscNB4g3Ex7xsXMMr1CjH47Tbh1iuQRCAaXeastx09gYWxf+Y0pYrhvvFsM89xtP35GfEyQ0g0edagMSj8n38QZyvfOw64D7degSQC0agCN5HroOxioXsy8fxEru8htpgdHG1/fkY8y8b2DsFdCnF8nAvMUikxOvA77WVHryCcFphJIhCNptvUZXYS6KR8xQL3X9DQGeKbwFbzZ462kgREuAnsFfzdP4hz1HKyAk4wG1tQEpL7OhGSCETQLd68i/Puf91eLHaVuplC95/xEcNg30Q+NZ1jAJIERDgKnEr6tnEpJUYH/uD6nz2DyKB2bCwcSCIQQTVh8YeMDvhNaJi2jKdiHuMLsx0DvZP5wjzL0V6SgAhndRMbFB7xD+Fs5Xtu1pba8bVlFWFRIpJEIIImc9bb9iBZDH4mu+YyJWYubxtpDPZNpJyWjvY9OrSSJCDCWuDEhvVGJ5bpGfzOtZhW7Levh0OJSBKBCIqLHnrDHg9oyw8UuifzW9cynvD3546asRygmaO9nCsgIkXgdOcZ/ptojpfRrpfsawZNfzqpJAJxUiYs/pDz7n+dn/Ta8416qyW8FvsQP1e+ZpTvHqb5hzq2jQBJAiKyTBnYGdfBg43LzPbM0/syVFtOJ+VLu01Tn04qiUCcECsBWN/gcfzEX1zP8Kz7Ub43W/Br31940+jmeI1C7TGTkgREpHks+1L7z7P82fxACx6JeRIN3b7elAeOJRGI49Zt6jLHbziXKaW87n6QodoKnvD359e+v1BmOs8OSIrV+HJGfzlTQESkgent6dGhFQD7iWdizc1con7Frdobdpu1ZRVNtkQkiUAcs6FPrndMC23Ffh51/Yf/xk7GrfgZWvMg0/xD8eJ2vO7MRLfsHSQiXmBPd4nRlbf0LoxxvUQHpW7W0LwNO5rkLCJJBKJBVgKwtolw4WeYtoyVseO4UXuHf/mvJ9P7KOuNToe9dlj3ZNlKWkSNwOmkD9WM4ACxzI553LE7aVOcReQK9Q2Ipmnx5l2MLSgh8MwlBYPr1Q2MdS3kPHU3G4yLmVBzC5+bZx/2+maaQunU607dDQvRBEwZ2JkFm3biN0zKacm4mjt51v1XJrjm8Sf/CKB2FlHmrLeb1NRpSQTCIfC8AIuGznXqRka5XqWjup1PjGRu8d3HKiONwNPELLJITESzx7IvtRdVvm2k8x9/f0a6XucT81zm630A+GyPp0klA0kEggmLPzzi9LYEDpCjrWaE603OVr6nzGjH3b7f8apxOeYRqoqqArNy0mRAWES1gentWfjeDvsXqkf9Q/i5sovJrrnsNNuwxqjdo+izPR66TV3WJEqnkgii0OLNu7hvYQlHOmtbwaCbWkq29jbXqZuIU3xsNC5iUs1wVhrpR0wAUFsble2jhaj1wu2Xc8GDb+A3THQ0/lDzB/7r/jP/ifkbI2ruY4PREYDdlT5SJy0J+WQKSQRR4Gi/8VsUDNKVz+mnvcuv1E0kq+XsN+N4Se9JgX4VHx6yU2ggKQMJcWSBJSIPceT5HuAF91TmxjzKnTVjWG3Urj3Y79U57/7XQ/rLlCSCCNLQB36gNuzlcvVjrlA/4mqthDOUffhMjfVGJ2b6sllqdOUnYo/6+liXyiODUqUMJMRRDExvz3vbK+yfye9pQa7vIfLdM3gm5lGm+3N5Sr8Oa5xt3oYdzN+4IyTlVUkETVh9JZzjEYuPC5WdXKJ+xSXKF/xC/ZQL1G8A2G82Z41xCUv1rqwy0qmkeb3vJdtDCHHsrN/wrWTwAy34jW8Sj8X8mwkxL9BD3cqf/CP42mwDgGHC6IISuydxqnoJDSYCwzB4+OGH+fTTT3G73UyZMoVzzz3XjhcWFrJgwQJcLhejRo3iqquuoqKignvvvZeffvqJM844g+nTpxMXF3dcbYPtSLNhIomGzuns52ylnPOV7zhf/ZbzlO/ooHzLBcouYpTape57zQQ2GxdQUNOb9UZHPjbPwziG5SQyBiDEiTk0GRygGXfV3MNw4y3+6FrAW+4/8oLeh6f817GbVo7Xztuw44i9/GD/QtZgIli+fDk+n4+CggJKSkqYMWMG//rXvwAoLy8nPz+fl156Ca/XS25uLj169GDOnDkMGDCArKwsnnjiCQoKCujfv/8xt7355puD9oDQ9JOAgoEbP7H4iMVPrFKDmxpiqSEOL0nKARI5QJJygCQOkKR4aIGHNsqPnKHs5UxlL23Yh6aY9nv6TZUd5hl8ZbZlpZHGh8bP2Gqez9dma4405fNIpPwjRHAcmgxA4Tm9H2/pl/HHmAXcoi3hFm0JG4yLWW504QOjA5+b7akkjiP9vK4tq2Dok+uDlgwaTATFxcX07NkTgLS0NLZu3WrHtmzZQnp6Om63G7fbTXJyMqWlpRQXFzNy5EgAevXqxaxZszjnnHOOuW2wE4GVBKa5nqKr+ikKJgq1H5qB/1Xs/1rXQAn4cLViylFeR+DrDn2fgNfWMlExcVODW6nbmOpY+EyN/cRTbrZgt9mKUiOZ3ZzGHrMl35in86XZjp1mG/wnUPlzqQqPZV8qH/5CBNmUgZ257NxWjoObvuV0xtT8jplKNkO0VVynbuThmOft+AEzln3E4zc1anCx0khnqn8YQFB/uW3wk6KqqoqEhAT7a03T8Pv9uFwuqqqqSExMtGPx8fFUVVU5rsfHx1NZWXlcbRvLl2ZbkkzPwa+sj+66j+fAr+2Pf/Pg1wcvBMYDX8chrzv0fZx/T+1/DRR8xOA1Y2r/Sww+XHiJwWu68eLCi5v9ZnN+JJ79ZnP2E4+XGI71t/pjIXV/IU6NgentGZjenm5Tl9l7dgF8bZ7BY/7BPEYObakgVf2CZGUPZyp7aYEHl6ITg58d5hmNcl8NJoKEhAQ8Ho/9tWEYuFyuI8Y8Hg+JiYn29WbNmuHxeEhKSjquto3lSX0AHN8v3xFJ6v1ChNbGhzKPMstP4TtO5zvj9FN6Pw2OEmZkZFBUVARASUkJKSkpdiw1NZXi4mK8Xi+VlZWUlZWRkpJCRkYGq1evBqCoqIguXbocV9tgs7aHjSYuVeHvg9P4akb/w/4nSUCI0JsysDNfzejvOOHseATzc00xTdOsr4E1a2jbtm2Ypsm0adMoKioiOTmZPn36UFhYSEFBAaZpMnLkSPr168f333/P+PHj8Xg8tGzZkpkzZ9K8efPjalufrKwsFi1adFwP2tQHjI+FlHCEiGzHOmX8RD4L6vvcbDARNEUnkgiEECKa1fe5KecRCCFElJNEIIQQUU4SgRBCRDlJBEIIEeUkEQghRJQLy91Hd+3aRVZWVqhvQwghwsauXbuOGgvL6aNCCCGCR0pDQggR5SQRCCFElJNEIIQQUU4SgRBCRDlJBEIIEeUkEQghRJSLyERgGAYTJ05k8ODB5OXlsX37dke8sLCQrKwscnJyWLVqVYjuMrgaeuZnn32W7OxssrOzmT17dojuMrgaemarzW233caLL74YgjsMvoaeefXq1eTk5JCdnc3DDz9MJMwOb+iZn3nmGbKyshg0aBDLli0L0V0G3wcffEBeXt5h11euXMmgQYMYPHgwhYWFwfnLzAi0dOlSc/z48aZpmubmzZvNO++8047t2bPHHDBggOn1es39+/fbfw539T3zjh07zBtvvNH0+/2mYRjm4MGDzU8++SRUtxo09T2zZebMmWZ2drY5f/78U317jaK+Z66srDT79+9v/vDDD6ZpmuYTTzxh/zmc1ffMP/74o3nllVeaXq/X3Ldvn9m7d+9Q3WZQPfHEE+aAAQPM7Oxsx3Wfz2f27dvX3Ldvn+n1es2srCyzvLz8pP++iOwRFBcX07NnTwDS0tLYunWrHduyZQvp6em43W4SExNJTk6mtLQ0VLcaNPU9c9u2bXnqqafQNA1FUfD7/cTGxobqVoOmvmcGWLJkCYqi2G0iQX3PvHnzZlJSUnjkkUfIzc2ldevWtGoV/qfz1ffMcXFxnHXWWVRXV1NdXY2iBO8s71BKTk7m8ccfP+x6WVkZycnJtGjRArfbTZcuXXj33XdP+u8Lyy0mGlJVVUVCQoL9taZp+P1+XC4XVVVVJCYm2rH4+HiqqqpCcZtBVd8zx8TE0KpVK0zT5NFHH6Vjx46cf/75Ibzb4Kjvmbdt28Zrr73GP/7xD/75z3+G8C6Dq75n3rt3Lxs3bmTx4sU0b96coUOHkpaWFvb/1vU9M0C7du3o378/uq4zcuTIUN1mUPXr14+vv/76sOuN9fkVkYkgISEBj8djf20Yhv1Nc2jM4/E4/o8NV/U9M4DX6+XBBx8kPj6eSZMmheIWg66+Z168eDG7d+9m+PDh7Nq1i5iYGNq3b0+vXr1CdbtBUd8zn3baaXTu3Jk2bdoAcNlll/HJJ5+EfSKo75mLiorYs2cPK1asAODWW28lIyOD1NTUkNxrY2usz6+ILA1lZGRQVFQEQElJCSkpKXYsNTWV4uJivF4vlZWVlJWVOeLhqr5nNk2Tu+66iwsvvJDJkyejaVqobjOo6nvmP/7xjyxcuJD8/HxuvPFGbr755rBPAlD/M3fq1Ilt27ZRUVGB3+/ngw8+4IILLgjVrQZNfc/cokULmjVrhtvtJjY2lsTERPbv3x+qW210HTp0YPv27ezbtw+fz8d7771Henr6Sb9vRPYIMjMzWbt2LUOGDME0TaZNm8bcuXNJTk6mT58+5OXlkZubi2majBkzJiLq5fU9s2EYbNq0CZ/Px5o1awAYO3ZsUL6BQqmhf+dI1NAzjxs3jttuuw2Aa6+9NiJ+yWnomdetW0dOTg6qqpKRkUGPHj1CfctB9+qrr3LgwAEGDx7M/fffz6233oppmgwaNIgzzzzzpN9fdh8VQogoF5GlISGEEMdOEoEQQkQ5SQRCCBHlJBEIIUSUk0QghBBRThKBEEJEOUkEQggR5f4fH3ijv89JMX8AAAAASUVORK5CYII=","text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":"p_grid = np.linspace(0,1,1000)     # 取1000个离散的参数值\nprior = np.repeat(1,1000)          # 20个参数值的不标准的先验分布\nlikelihood = binom.pmf(6,9,p_grid) # 投掷9次，6次海面朝上的各参数的似然\nunstd_posterior = likelihood*prior # 没有归一化的后验分布\nposterior = unstd_posterior/np.sum(unstd_posterior) #后验分布归一化\n\nplt.plot(p_grid,posterior,'o')#画出1000个值的点\nplt.plot(p_grid,posterior)#将1000个值的点进行连线"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"B7A370C25E174B9A9D9C60A6B45E7FD0","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"从以上代码可以看出：grid approximation 其实就是将连续参数求积分的过程转化为离散参数求和的过程"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"64FBCF3FDC914FEA9B6F6128483E3FAA","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"当只存在一个参数时，grid approximation 时很有用的，但是当存在两个以上参数时， grid approximation就需要大量计算\n\n假如只有一个参数时，grid approximation计算时取100个参数值，则当存在2个参数时，就需要计算 $100^2=10000$个参数值"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"1ACFAB4E4EED44068D1768EC3A67E97C","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"### 马尔科夫链蒙特卡洛(markov chain monte carlo, MCMC)"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"0D121F7C68D940C08467A6B9F7BE2AC4","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"在计算具有多个参数的后验分布时，有一种有效的方法叫做马尔科夫链蒙特卡洛(MCMC)。\n\n在介绍MCMC之前，我们先分别介绍一下蒙特卡洛方法(monte carlo)和马尔科夫链(markov chain)。"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"A9033512BAD04CF896E93F491B623DC5","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"#### 蒙特卡洛方法"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"112B933FC473494A8936284B5FC0E2D2","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"蒙特卡洛方法来自于摩纳哥的蒙特卡洛赌场，许多纸牌类游戏需要计算其胜利的概率。\n\n我们可以将蒙特卡洛理解为简单的模拟，通过模拟的情景来计算其发生的概率。\n\n因为许多概率的计算往往是非常复杂甚至是无法直接计算的，但是我们可以通过计算机对期待发生的场景进行大量模拟，从而计算出其发生的概率，即：\n\n$p(发生的概率)=\\frac{模拟情形下事件发生的次数}{模拟的次数}$"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"E97ED72D3547458A8E85A6922D9440E8","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"我们使用一个经典的例子来说明蒙特卡罗方法的原理。\n\n我们可以用蒙特卡洛方法来求出不规则图形的面积。\n\n其基本原理为用大量的随机点铺满整块区域（该区域为规则图形），其中不规则图形的面积为\n$\\frac{落在不规则图形内的点的数量}{该区域内点的数量}*该区域面积$"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"5E548D5A2A74426A8110E7D464CFA7A0","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"我们该如何计算图形下半部分面积？"},{"cell_type":"code","execution_count":4,"metadata":{"collapsed":false,"id":"1708BEF2858C44FA9B0B5CDC50C5F641","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"output_type":"execute_result","data":{"text/plain":"(-2.0, 2.0)"},"metadata":{},"transient":{},"execution_count":4},{"output_type":"display_data","data":{"text/plain":"<Figure size 432x288 with 1 Axes>","text/html":"<img src=\"https://cdn.kesci.com/upload/rt/1708BEF2858C44FA9B0B5CDC50C5F641/rjw3d9udk1.png\">"},"metadata":{},"transient":{}}],"source":"x = np.linspace(-2,2,400)     # 取-2到2间的400个点\ny = x**3+2                    # 通过非线性关系来获得y的取值   \nplt.plot(x,y,color = 'black')\n\nn = 100000                         # 随机点的数量\nuniform_x = uniform(-2 , 4).rvs(n) # 从-2到2之间随机取100000个点\nuniform_y = uniform(-8, 16).rvs(n) # 从-8到8之间随机取100000个点\nplt.plot(x,y)\nplt.gca().axes.set_ylim(-8, 8)     # 将x轴限定在-8到8\nplt.gca().axes.set_xlim(-2, 2)     # 将y轴限定在-2到2"},{"cell_type":"code","execution_count":7,"metadata":{"collapsed":false,"id":"E03A455616F04643980B2758622F8BA1","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"output_type":"stream","name":"stdout","text":"图形下半部分面积为39.81824\n"},{"output_type":"display_data","data":{"text/plain":"<Figure size 432x288 with 1 Axes>","text/html":"<img src=\"https://cdn.kesci.com/upload/rt/E03A455616F04643980B2758622F8BA1/rjw3fy4jx3.png\">"},"metadata":{},"transient":{}}],"source":"count = 0     # 落在目标区域的点的数量\ntarget_x = [] # 落在目标区域的点的x值\ntarget_y = [] # 落在目标区域的点的y值\n\nfor i in range(n):# 对每一个点做一次判断\n    if  uniform_y[i] <= uniform_x[i]**3+2:  # 击中点的特征，y<= x^3 +2\n        count += 1                          # 击中点的数量+1\n        target_x.append(uniform_x[i])       # 将击中点的x值加入到击中点x值的集合中\n        target_y.append(uniform_y[i])       # 将击中点的y值加入到击中点y值的集合中\n\nplt.scatter(uniform_x, uniform_y) # 绘制全部的点的图\nplt.scatter(target_x,target_y)   # 绘制目标点的图\nplt.gca().axes.set_ylim(-8, 8)\n\ns = (count/n)*(4*16) #计算不规则图形面积\nprint(f\"图形下半部分面积为{s}\")"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"E562BC306A884D35AB664E6D6EFA4C81","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"可以将蒙特卡洛理解为一种模拟事件发生的方法。"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"48DA3F27FD024F2A8C68A3C6B5D7DABE","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"使用蒙特卡洛方法进行采样时，常见做法是：接受-拒绝采样。\n\n为了求解一个不常见的分布$p(x)$，先构造一个常见的分布$q(x)$进行采样，然后按照一定的方法拒绝某些样本，以达到接近 $p(x)$ 分布的目的，其中$q(x)$叫做 proposal distribution(建议分布)。"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"8FF6A03E0F8440ED92E5A18EBFDE22F3","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"假定我们有一个不常见的概率分布$p(x)$，已知该分布的先验为均匀分布，似然函数已知，想要计算该参数的后验分布"},{"cell_type":"code","execution_count":18,"metadata":{"collapsed":false,"id":"DFCBC37AD4E040799EBD508C8675C91C","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"output_type":"execute_result","data":{"text/plain":"[<matplotlib.lines.Line2D at 0x7f7482b26410>]"},"metadata":{},"transient":{},"execution_count":18},{"output_type":"display_data","data":{"text/plain":"<Figure size 432x288 with 1 Axes>","text/html":"<img src=\"https://cdn.kesci.com/upload/rt/DFCBC37AD4E040799EBD508C8675C91C/rjw4put83a.png\">"},"metadata":{},"transient":{}}],"source":"x = np.arange(-4, 6, 0.01) # 生成从-4到6的数列\n\n# 定义一个不常见的分布\ndef p_func(x):\n    res = (0.3 * np.exp(-(x - 0.3) ** 2) + 0.7 * np.exp(-(x - 2) ** 2 / 0.3)) /1.21\n    return res\n\nplt.plot(x, p_func(x), color = 'r', lw = 5, label = 'p_func(x)') #画分布的形态"},{"cell_type":"code","execution_count":10,"metadata":{"collapsed":false,"id":"EC88CA6AE3DF4DF68CB1A2443C765C23","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[],"source":"norm_rv = norm(loc = 1.4, scale = 1.2) # 建议分布\nc = 3 # 使建议分布的每个值大于目标分布的参数"},{"cell_type":"code","execution_count":12,"metadata":{"collapsed":false,"id":"C0A4183FD98A46DC8E13E8B15CB346A1","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"data":{"text/plain":["[<matplotlib.lines.Line2D at 0x26b1a648ac0>]"]},"execution_count":12,"metadata":{},"output_type":"execute_result"},{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAXIAAAD3CAYAAAAALt/WAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAlSklEQVR4nO3deVxU5f4H8M8AIuiYuG9FGYamV0PUny2SC5GmaYoLSJL+1Fu2WTdSu3bjeqmLWt1+7Wneq2nXEPfUShOXVCwXBMty16hc0lyiIWWb8/vjUWLmPDgKM8+ZM/N5v168gO8h+Gry4fCcZ7FomqaBiIhMK8DoBoiIqHoY5EREJscgJyIyOQY5EZHJMciJiEwuSPUX7Nq1K1q0aKH6yxIRmdqxY8ewbds26TXlQd6iRQssXbpU9ZclIjK1+Pj4Sq9xaIWIyOQY5EREJscgJyIyOQY5EZHJMciJiEzuqoJ89+7dSE5O1tXXr1+PwYMHIyEhAQsXLnR7c0TeyG4HDh0CsrOBbduAX34xuiPydy6nH86aNQsrVqxAaGioQ72kpARTp07F4sWLERoaiuHDh6NXr15o2LChx5olMtKXXwLvvQd8/DFQUOB4LTISSEoCHn0UaNzYmP7If7m8Iw8PD8dbb72lqx8+fBjh4eGoW7cugoOD0alTJ+zYscMjTRIZ6fhxYOBA4M47gQ8/1Ic4ABw4AEyZAkREAK+8ApSVqe6S/JnLIO/duzeCgvQ37jabDXXq1Cl/v3bt2rDZbO7tjshga9YAt90m7sKvhs0GTJwI3HMPh1xInSo/7LRarSgsLCx/v7Cw0CHYicxu/nygX7+qBfLGjUC3bkB+vtvbItKpcpBHREQgPz8f58+fR3FxMXbu3ImOHTu6szciw3z4IZCcXPkQidUKdOkCtGsHWCzyj9m/H+jeXQzNEHnSNQf5ypUrkZmZiRo1auC5557DmDFjkJiYiMGDB6NJkyae6JFIqQ0bgNGjAdkhiE2aALNnA6dPA9u3A3v2AD/8ADz2mDzQ8/OBvn3l4+pE7mJRfWZnfHw8N80ir3X0KBAdDZw/r7/WsyeQkSHCXGbdOmDwYODXX/XXBg0Cliyp/O6dyJUrZScXBBFdUlIiphDKQnzAAODTTysPcQCIjQU2bwYaNdJfW7YMePNNt7VK5IBBTnRJWhrw1Vf6erduwMKFQEiI68/Rvj3wySdArVr6axMmiKEYIndjkBMB2LkTSE/X12+8EVi+HKhZ8+o/V5cuwLx5+npJCTB2LOeYk/sxyMnvlZUB48aJpfcVBQQAH30ENGhw7Z9z8GBg/Hh9fds2QLK+jqhaGOTk92bMAHJy9PXUVLGas6qmTwfatNHX//Y3Tkkk92KQk187exZ4/nl9/U9/AiZPrt7nDgkBZs3S1wsLgRdeqN7nJqqIQU5+bepU+XTBGTOAGjWq//m7dRNzzJ3NmQPk5VX/8xMBDHLyYz/9JB+v/t//Be66y31f58UXgXr1HGuaJmaxELkDg5z81pQpQFGRYy00FHjpJfd+nfr15UMpWVnAli3u/Vrknxjk5JcOHhTDG86eegpo3tz9X+/xx8UWt87+8Q/3fy3yPwxy8ksvv6yfbhgWJrag9YTgYPEbgLOsLGDrVs98TfIfDHLyO8eOAXPn6uuTJunHst0pMRG45RZ9nXflVF0McvI7r70mVllWFBYmhj88KShIzCF39vnnnMFC1cMgJ79y9iwwc6a+/sQTgIpzUZKSgFat9PX/+z/Pf23yXQxy8ivvvy8W5FQUGipfTu8JQUHycfiMDK72pKpjkJPfKC0F3n1XX//zn+Vbz3rKiBFAw4aOtZIS4J131PVAvoVBTn5jxQrgxx8daxYL8PTTavsIDZWv9nzvPf1vC0RXg0FOfkO2irN/f6BlS/W9PPaYmJJY0blzQGam+l7I/Bjk5Bf27BEn2zt78knlrQAQJw09+KC+PmOG+l7I/Bjk5Bfefltfa9NGHM9mFNnwyo4dwK5d6nshc2OQk88rLATmz9fXn3jC2MOQO3cGOnXS12XTI4muhEFOPm/RIsBmc6xZrcBDDxnTT0WPPKKvzZ8PFBSo74XMi0FOPu8//9HXhg9XswDIFVkflf0GQVQZBjn5tAMH5FvFjh6tvhcZq1XMK3c2e7b6Xsi8GOTk02SB2LYt0LWr+l4qIxte2bkT+O479b2QOTHIyWeVlsp3ORwzxtiHnM5uuw3o2FFfl/VOJMMgJ5/12WfAyZOOtaAg+VCG0UaO1Nf++1+grEx9L2Q+DHLyWfPm6WsDBgCNG6vvxZWkJPFDpqLjx8XBE0SuMMjJJ/36K7Bypb7uLQ85nTVqBPTtq69zeIWuBoOcfNLSpfqDlRs1Anr3NqafqyEbXlm2TPxQIroSBjn5JNk87IQE/fCFN+nXD6hf37F28SKwZIkx/ZB5MMjJ55w4Aaxfr68nJanv5VrUrCkWCDlbsEB9L2QuLoPcbrcjNTUVCQkJSE5ORn5+vsP12bNnIz4+HoMHD8batWs91ijR1VqwANA0x9rNNwO3325MP9dCNqNm3Trg1Cn1vZB5uAzyrKwsFBcXIzMzEykpKZg2bVr5tYKCAsybNw8LFizA7NmzkZ6e7tFmia6GbFglKcm75o5XpmtX4MYbHWt2O7B4sTH9kDm4DPKcnBzExMQAAKKiorBnz57ya6GhoWjevDkuXLiACxcuwGKG7xTyafv3Azk5+rq3D6tcZrEAiYn6OodX6EpcBrnNZoPVai1/PzAwEKWlpeXvN2vWDP369cOgQYPwkDdsJ0d+LSNDX+vYEbj1VvW9VJUsyDdv1h9TR3SZyyC3Wq0orHCQoN1uR9ClR/+bNm3CqVOnsG7dOmzcuBFZWVn4+uuvPdctkQsLF+prspN4vNltt4lDL5zJ/mxEwFUEeXR0NDZt2gQAyMvLQ2RkZPm1unXrIiQkBMHBwahZsybq1KmDAm6kTAb57jtg7159PSFBfS/VweEVulYuZ9XGxcUhOzsbiYmJ0DQN6enpmDNnDsLDwxEbG4utW7di2LBhCAgIQHR0NO666y4VfRPpLFqkr915J3D99ep7qa6EBGDKFMfazp3AoUNAq1aGtERezGWQBwQEIC0tzaEWERFR/vb48eMxfvx493dGdI1kMzuGDFHfhzu0aQNERQF5eY71hQuByZON6Ii8GRcEkU/Ytw+oMKGq3ODB6ntxF9nwCld5kgyDnHyC7G68a1cgPFx9L+4i+21i1y7g+++Vt0JejkFOPsGXhlUui4gQM1icLV2qvhfybgxyMr2DB4Hdu/V1Mw+rXCb7M3B4hZwxyMn0ZHfjnTsDLVuq78XdZEG+das4dILoMgY5mZ4vDqtc1ratfHHQ8uXKWyEvxiAnUztyRDwAdOYLwyqXcXiFXGGQk6nJAi0qyrcWzcTH62tffAH88ov6Xsg7McjJ1JYt09eGDlXfhyd17AjcdJNjrawM+PhjQ9ohL8QgJ9M6eRL46it9XXYHa2YWi3x4hdMQ6TIGOZnWypX6k4AiI+UPB81OFuRr1/JgZhIY5GRasqGFgQOVt6FE165A8+aOtZISYNUqY/oh78IgJ1Oy2YCsLH39gQfU96JCQAAwaJC+znFyAhjkZFKffw4UFTnWGjcWd66+Shbkn32m/3sg/8MgJ1OSLYjp3x8IDFTeijJ33w2EhTnWbDZgwwZD2iEvwiAn0yktBT75RF/31fHxy2rUAPr21dc5vEIMcjKdLVuAs2cda7VqAbGxxvSjkuwZwIoVgN2uvhfyHgxyMh3ZsErv3kBoqPJWlOvTR9yZV3T8OJCTY0w/5B0Y5GQqmuZf0w6dXXcd0KuXvs7hFf/GICdT+eYb/Qk5gYFAv36GtGMI2fAKg9y/McjJVGSB1a0b0KCB+l6MMmCAvrZnj9gJkvwTg5xMRTY+7quLgCrTooU4OMMZ78r9F4OcTOPHH+V7j/tbkAMcXiFHDHIyjRUr9LX27YGbb1bfi9FkwyubNwNnzqjvhYzHICfTkN1x+uPdOCB+gDnvUW63yxdKke9jkJMpnD8vX4rur0FusXB4hf7AICdT+OwzsTS/ohYtgE6djOnHG8iCfM0a4OJF9b2QsRjkZAqVDatYLOp78RYxMUC9eo61wkJg3Tpj+iHjMMjJ6xUVAZ9+qq/767DKZUFB8oVQHF7xPwxy8nobNwK//eZYu+46oEcPI7rxLrIfZitXchMtf8MgJ68nu8Ps2xcIDlbfi7fp3Vv/93DyJLB9uzH9kDFcBrndbkdqaioSEhKQnJyM/Px8h+tffPEFhg0bhqFDh2LKlCnQnE/DJaoGu10+f9zfh1Uuq1NHvn0vh1f8i8sgz8rKQnFxMTIzM5GSkoJp06aVX7PZbHjllVcwY8YMLFq0CC1atMC5c+c82jD5l5wc4Ngxx1qNGsB99xnTjzfiNERyGeQ5OTmIiYkBAERFRWHPnj3l13JzcxEZGYnp06cjKSkJDRs2RP369T3XLfkd2d14jx5A3brKW/Fa/fvra3v3AgcPqu+FjOEyyG02G6xWa/n7gYGBKL00offcuXPYtm0bnn32WcyaNQtz587F0aNHPdct+R2u5nSteXP5odO8K/cfLoPcarWisLCw/H273Y6goCAAQFhYGNq3b49GjRqhdu3a6Ny5M/bu3eu5bsmvHD0q9h93JttnxN9xeMW/uQzy6OhobNq0CQCQl5eHyMjI8mvt2rXDgQMHcPbsWZSWlmL37t1o1aqV57olvyILouho4IYb1Pfi7WRBnp0NnDqlvhdSL8jVB8TFxSE7OxuJiYnQNA3p6emYM2cOwsPDERsbi5SUFIwdOxYA0KdPH4egJ6oOzla5erfeCrRqBRw69EdN04BVq4DRo43ri9RwGeQBAQFIS0tzqEVERJS/3a9fP/Tzp3O2SImzZ4FLvwg6YJDLXd5E61//cqx//DGD3B9wQRB5pU8/BcrKHGs33gh06GBMP2Yg+yG3di3w++/qeyG1GOTklWTj4wMG+PcmWa7ceSfQsKFj7cIFEebk2xjk5HWKioDVq/V1DqtcWWCgfE45Z6/4PgY5eZ0NGwCbzbEWFgbcfbch7ZhKZZtoOQ9TkW9hkJPXqWyTrBo11PdiNnFxQGioY+2XX4CtW43ph9RgkJNX4SZZ1VOrlghzZxxe8W0McvIqOTnA8eOOtRo1gD59jOnHjCpb5cmNSX0Xg5y8iuxuvGdPcZAEXZ3+/YEAp+/sQ4fERlrkmxjk5FW4SVb1NWokpiI64/CK72KQk9fgJlnuI/vht3y58jZIEQY5eQ3ZHWOnTsD116vvxexkQb59u/75A/kGBjl5jcpWc9K1u+UWsZGWs5Ur1fdCnscgJ69w9iywebO+zvHxquMe5f6DQU5egZtkud/AgfraunXAb78pb4U8jEFOXkH2IO6BB7hJVnV06QI0a+ZYKy4G1qwxph/yHAY5Ge7334HPPtPXOaxSPQEB8k20OHvF9zDIyXBr1uj3zK5fn5tkuYPsh+EnnwAlJep7Ic9hkJPhli7V1x54AAhyeX4VudKrF1C7tmPt/Hn5g2UyLwY5Gaq4WD4lbvBg9b34opAQ+T41nL3iWxjkZKj164Fff3Ws1akDxMYa048vks1eWbpU7DRJvoFBToaSDav06yfuJMk9+vXTD1P99BOwY4cx/ZD7McjJMGVl8hkU8fHKW/Fp9eqJsXJnS5ao74U8g0FOhsnOBk6fdqyFhAD33WdMP75syBB9bfFi7lHuKxjkZBjZHWHv3oDVqr4XXzdwoH6P8qNHgdxcQ9ohN2OQkyE0TT4+zmEVz2jUCOjeXV9fvFh9L+R+DHIyxM6d4oFbRUFBwP33G9OPP+Dwiu9ikJMhZHfjPXuKFZ3kGYMG6feuOXgQ2LPHmH7IfRjkpJymyX+l5yIgz2rWDOjWTV/n8Ir5MchJudxccRhwRRYLN8lSQfbDkkFufgxyUi4zU1+7+26gaVP1vfgb2cPk774D9u5V3wu5D4OclNI0YOFCfT0hQX0v/uiGG4Dbb9fXuTjI3BjkpNSOHcD33zvWAgI47VAl2eyVRYvU90Hu4zLI7XY7UlNTkZCQgOTkZOTn50s/ZuzYscjIyPBIk+Q7ZMMqPXsCTZqo78VfyX5ofv01sG+f+l7IPVwGeVZWFoqLi5GZmYmUlBRMmzZN9zGvv/46CgoKPNIg+Q67ncMq3qBlS6BzZ32d92Hm5TLIc3JyEBMTAwCIiorCHqdJp6tXr4bFYin/GKLKfPmlfBEQh1XUS0zU1zIyuDjIrFwGuc1mg7XC5heBgYEoLS0FABw4cACrVq3CU0895bkOyWfI7sbvuQdo0EB9L/4uIUG+OGjXLmP6oepxeZiW1WpFYWFh+ft2ux1BlzY3Xr58OX7++WeMHDkSx44dQ40aNdCiRQvczcMWyUlZmfyBGodVjHH99WLK5xdfONYzMoBOnYzpiarOZZBHR0djw4YN6Nu3L/Ly8hAZGVl+beLEieVvv/XWW2jYsCFDnKS2bAFOnHCsBQfLT68hNYYP1wf5ggXAyy/rd0ok7+byf1dcXByCg4ORmJiIqVOn4q9//SvmzJmDdevWqeiPfITsQVrv3kBYmPJW6JIhQ/QnBx07xoOZzcjlHXlAQADS0tIcahEREbqPe/LJJ93XFfmUoiL5+PiwYep7oT80aADcey/w6aeO9YwM+Za35L34CxR53CefAOfOOdZq1eKwijcYPlxfW7wYKClR3wtVHYOcPO6//9XX4uN5EpA3eOABIDTUsXbmDLB2rTH9UNUwyMmjzp4FVq3S10eMUN8L6dWpA/Tvr6/Pn6++F6o6Bjl51KJF+l/TmzYFYmON6Yf0ZMMry5YBXKxtHgxy8qgPP9TXhg/Xz5Yg49x3n/5kpgsXuJGWmTDIyWOOHAGys/X15GT1vVDlataU35V/8IHyVqiKGOTkMbJx1rZtgago5a2QC6NG6WtbtuhPciLvxCAnj9A0YN48fT05Wb/HBxmvUyegXTt9Xfb/kLwPg5w8YtMm+d1cUpL6Xsg1iwUYOVJfnztXbD9M3o1BTh7xn//oa7GxQHi4+l7o6owYod9j5Ycf9PuxkPdhkJPbnT8vP5l9zBjlrdA1aNZM7H/jjA89vR+DnNwuI0NMX6uoXj1g0CBj+qGrJ3vouXgx8Ouvyluha8AgJ7eTDauMGAGEhKjvha7NgAH6HSl//50rPb0dg5zcKi8PyMnR18eOVd4KVUFICPDgg/r6zJk8Bs6bMcjJrWR34507Ax06qO+FquaRR/S1r78Gtm1T3wtdHQY5uU1hoXynQz7kNJf27YE77tDXZ85U3wtdHQY5uc1HH4kZKxWFhsqXf5N3k92VL1ig31eevAODnNxC04B33tHXk5KAunXV90PVM2yY/qHnxYvyTdDIeAxycoutW4Hdu/X1xx9X3wtVX2go8NBD+jofenonBjm5xdtv62t33AF07Ki+F3IP2fDKd98B69er74WujEFO1XbyJLBkib7Ou3Fza9sWiInR1994Q30vdGUMcqq2WbP0pwA1bgwMGWJMP+Q+48fra6tWAQcPqu+FKscgp2opLgZmzNDX//xncWABmdvAgfqNzjQNeOstQ9qhSjDIqVo++gg4ftyxFhAgH18l8wkKAp58Ul+fPVs/1ZSMwyCnKtM04NVX9fX4eOCGG9T3Q54xZgxQu7ZjrbBQhDl5BwY5Vdnq1cC33+rrzz6rvhfynHr15LsivvkmUFqqvB2SYJBTlcnuxmNigK5d1fdCniV76JmfD2Rmqu+F9BjkVCW7dsnnE/Nu3DdFRgL9+unrU6fyKDhvwCCnKpk2TV9r3Rq4/371vZAazz2nr337LbBypfpeyBGDnK7Zt9/Kj3JLSdGf+Ui+o1s34O679fX0dC7bNxq/7eiavfSS/hu3aVMgOdmYfkidyZP1te3buWzfaAxyuiZ798ofcD33HI9y8wf33gtER+vraWm8KzeSyyC32+1ITU1FQkICkpOTkZ+f73D9gw8+wNChQzF06FC8Lds5iXxKZXfjDz9sTD+klsUivyvftAnIylLfDwkugzwrKwvFxcXIzMxESkoKplV4yvXjjz9ixYoVWLBgARYuXIgtW7Zg3759Hm2YjLNvnzhcwNnEiWLbU/IPgwaJDbWcTZ7Mu3KjuAzynJwcxFzaAi0qKgp79uwpv9a0aVP8+9//RmBgICwWC0pLS1GTG2z4rOef1081a9KEy/H9TUCAGEpxtnMnsHy58nYIVxHkNpsNVqu1/P3AwECUXlrOVaNGDdSvXx+apmH69Olo27YtWrZs6bluyTBbtwJLl+rrEyYAtWqp74eMFR8vHyt/4QWgrEx9P/7OZZBbrVYUFhaWv2+32xEUFFT+flFREZ599lkUFhbi73//u2e6JENpmhg+cdaiBfDoo+r7IeNZLMA//6mvf/ut/ABu8iyXQR4dHY1NmzYBAPLy8hAZGVl+TdM0PPbYY2jdujXS0tIQGBjouU7JMCtWANnZ+npaGu/G/Vnv3mJuubPJkwGbTX0//izI1QfExcUhOzsbiYmJ0DQN6enpmDNnDsLDw2G327F9+3YUFxdj8+bNAIBnnnkGHXm+l88oKZGv6GvXDhg5Un0/5D0sFrEYyHmR0PHjwPTpwIsvGtOXP3IZ5AEBAUhzerIRERFR/vY333zj/q7Ia7z5ppit4mz6dIC/gFFMjDh8wvkh56uviu1vb7rJgKb8EBcEUaWOHwemTNHXu3cH+vZV3g55qVdfBYKDHWsXL8qfq5BnMMipUs8+qx/rtFiA114Tr4kAICICePppfX3RIi4SUoVBTlIbNgAZGfr6uHHyaWfk355/XqwpcPbII8Dvv6vvx98wyEmnsBAYO1Zfb9BALNEncnbddWJvcmdHjsiH58i9GOSk87e/iW9AZ9OmAfXrq++HzGHkSPk2t6+9Jg4iIc9hkJODrVuBN97Q1++8Exg9Wn0/ZB4BAcD77wPOu3SUlYkZLMXFxvTlDxjkVM5mE4fsOm98FBIiTkznoRHkSuvW4jc6Z3l5Yvk+eQa/NancE08ABw/q62lp4huU6GpMnAj86U/6+iuv8AAKT2GQEwBg/nxg7lx9vUsX4C9/Ud8PmVdwMPDBB0CQ03JDTQMeegg4c8aQtnwag5xw4IB886tatYB58/TfkESudOokX6J/7BiQlMQdEt2NQe7nCgqABx4AfvtNf+3tt4E2bdT3RL5hwgSxCtjZ558Dqanq+/FlDHI/ZrcDI0bI91JJTBQPPomqKjAQ+PBDoF49/bX0dGDZMvU9+SoGuR/761+BlSv19VatgBkzuAyfqu+GG4CPPpL/W3rwQWDbNvU9+SIGuZ964w3g5Zf1datV7GRXt67ylshH9ekjPxruwgWgf3/g8GH1PfkaBrkfWrBAvskRIE53addOaTvkByZPBgYM0NdPnxYHVBw/rr4nX8Ig9zOLFwPJyfJraWniwSeRuwUEiJsE2Zkzhw8DvXoBJ06o78tXMMj9SGameIh56exsBw8/LF+RR+QudeoAn3wC3Hij/tr+/Qzz6mCQ+4lZsyqfvztwIPDOO3y4SZ7XrBnw2Wfyzdf27RN7+uzfr74vs2OQ+zi7XYxPPvyweNtZr15iVgEX/ZAqt94KrF0LhIXpr33/PXDXXcCXX6ruytwY5D6soABISJDvEw0AcXFi+mFoqNq+iKKjRZjLZkedOQP06CF2UnTewI3kGOQ+avdusUx68WL59fvuA1asEMvwiYzQubMI84YN9deKi8XpQmPHimmKdGUMch9TViY28r/9duDQIfnHjBol5oqHhKjsjEivSxcgOxu46Sb59dmzxUwXLhy6Mga5D/nuOzG+mJIiTjGX+cc/xDeH86nnREaJjBRj4p07y6/v3y8egk6aJI4hJD0GuQ/45Rdg/Hjgttsqv3OpU0dMP0xN5ewU8j5NmwKbN4uThGTsdrESuXVrseUyx84dMchN7Nw5cRhyq1bAW2/J54cDQIcOwM6dwLBhavsjuhYhIcC//y2myjofF3fZsWNio7fOncWmW7KZWP6IQW5CP/wgtggNDxfHZ/36q/zjLBbgySeBr74Sv74SmcHYseKw5i5dKv+YXbuA+HggKkociOLvD0QZ5CZx8aKYgdKnj3gw9Oqr4ozNyrRuLX5VffNNTi8k82nbVhwE/tJLV36e88034uF98+Zi/6Bdu/xz2IVB7sXOnwcyMsSQSKNGwNChwJo1V/6HarWKf/y5ueLBJ5FZBQUBzz8vHuIPHHjljz1/Xuzo2amTGGqcNEnMhikpUdGp8biez0tomtgBbudO4IsvgI0bxcnjV3t3ERgIjB4tNr5q2tSTnRKpFREhxsOzskSwb99+5Y8/ckQ8GH35ZXFjc/fdYgVz165iKMZqVdK2Ugxyxex28cDm0CGx69vBgyKwc3PFlp7XKiREjCmmpFQ+F5fIF9xzDxAbKxYRvfgisGWL6//GZgM+/VS8AOK5UZs2Ym5669bi2dEtt4iX667zbP+exCCvhrIy8Q+loED/cu4c8PPP4uXkyT9efvoJKCqq/tdu2VLcgT/8MNC4cfU/H5EZWCzAvfeK7SW2bwdmzhT761/tw05NA/buFS/O6tUTY+3NmonXzZuL762wMP3LddeJZ08hIWKLXqO5DHK73Y4pU6Zg//79CA4OxksvvYQbK+xDuXDhQixYsABBQUF49NFH0bNnT7c3WVwsxop37hRT7Ox28T/k8mt3v11cLF6KihxfO9fcEcjXIixMnKgyapTYi8Ib/gERGcFiEUMlXbuKlcwZGWIywMaNVZ+SeO6cePn222v774KDRahfDvbLb9eoIcb5g4LE2zVrAnfcIR7K1q5dtR4rpbmwZs0abdKkSZqmaVpubq42bty48munTp3S7r//fq2oqEgrKCgof/tKBg0a5OpL6gwZcjlm/e/llls07bHHNG3tWk0rLr7mvzoiv/Lzz5o2c6amDRqkaWFhxn//yl769avan+1K2enyjjwnJwcxMTEAgKioKOzZs6f82tdff42OHTsiODgYwcHBCA8Px759+9ChQwe3/aA5caLyjZ98TXAw0L692BmuRw+ge3egRQujuyIyj8aNxXDjww+Loc/cXGD9emDHDiAnBzh61OgOxeEaZ8/K92SvKpdBbrPZYK3wmDcwMBClpaUICgqCzWZDnTp1yq/Vrl0btitNbq6CmjXFEIIvreCqW1dMkYqIEK8jI8XDl1tvFb+CEVH1BQaKFaAV93A5e1aE+759wIEDYrLBwYMi4GWHrnhCnTrunznjMsitVisKK+xUY7fbEXTpFALna4WFhQ7B7g7164tVjNOnu/XTuo3VKh58yF4aNxZTAZs0Ea8vv9Srx/1OiIxQv76Y+RIb61gvKwNOnRJTgE+cEK+PHxdj5ufP618KCsQD1mt9ThYQIBbpuXvTOpdBHh0djQ0bNqBv377Iy8tDZIW13h06dMDrr7+OoqIiFBcX4/Dhww7X3WXqVGDwYPGrkaaJELRYxF+K89uy2rW+HRwsXmrWlL+u+HZgoNv/uESkWGCgmK3SrNm1/Xd2uwjzixdFsFd8XVIiJmeUlPyxMCk6Wr7/enW5DPK4uDhkZ2cjMTERmqYhPT0dc+bMQXh4OGJjY5GcnIykpCRomoa//OUvqFnZbjfVYLGIfReutPcCEZFqAQF/zFKpV8+4PiyapnZngvj4eCxdulTllyQiMr0rZSdnIhMRmRyDnIjI5BjkREQmxyAnIjI5BjkRkckp3/3w2LFjiI+PV/1liYhM7dixY5VeUz79kIiI3ItDK0REJscgJyIyOQY5EZHJMciJiEyOQU5EZHIMciIikzNlkB8+fBidOnVCkerTjw3w22+/Ydy4cRgxYgQSEhKQm5trdEseY7fbkZqaioSEBCQnJyM/P9/oljyupKQEEyZMQFJSEoYMGYJ169YZ3ZISZ86cQffu3XH48GGjW1Fm5syZSEhIQHx8PBYtWuTWz618QVB12Ww2TJ8+HcHuPmLDS82ZMwe33347Ro0ahSNHjiAlJQXLli0zui2PyMrKQnFxMTIzM5GXl4dp06bhvffeM7otj1qxYgXCwsLwyiuv4Pz58xg4cCBinY+v8TElJSVITU1FSEiI0a0os23bNuTm5iIjIwMXLlzA7Nmz3fr5TXVHrmkaXnjhBTzzzDMIDQ01uh0lRo0ahcTERABAWVmZRw7u8BZXOujbV/Xp0wdPPfUUAPHvO9APjpyaPn06EhMT0bhxY6NbUWbLli2IjIzE448/jnHjxqFHjx5u/fxee0e+aNEizJ0716HWvHlz9O3bF23atDGoK8+S/ZnT09PRoUMHnD59GhMmTMDkyZMN6s7zrnTQt6+qXbs2APFnHz9+PJ5++mljG/KwpUuXon79+oiJicH7779vdDvKnDt3DsePH8eMGTPw008/4dFHH8Xq1athcdPhvV77HTJ06FAMHTrUoRYXF4clS5ZgyZIlOH36NEaPHo358+cb1KH7yf7MALB//34888wzmDhxIv7nf/7HgM7UuNJB377sxIkTePzxx5GUlIT+/fsb3Y5HLVmyBBaLBV9++SX27t2LSZMm4b333kOjRo2Mbs2jwsLCcPPNNyM4OBg333wzatasibNnz6JBgwbu+QKaSfXs2VO7ePGi0W143MGDB7XevXtre/fuNboVj1u9erU2adIkTdM0LTc3VxszZozBHXne6dOntT59+mhbt241uhXlRowYoR06dMjoNpRYv369NmrUKM1ut2snT57U7rnnHq20tNRtn9/3b3dM7l//+heKi4vxz3/+E4C4a/XVB4Cyg7593YwZM1BQUIB3330X7777LgBg1qxZfvUg0B/07NkTO3bswJAhQ6BpGlJTU936PIS7HxIRmZypZq0QEZEeg5yIyOQY5EREJscgJyIyOQY5EZHJMciJiEyOQU5EZHL/D/tBAje1x8YgAAAAAElFTkSuQmCC","text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":"x = np.arange(-4, 6, 0.01)\nplt.plot(x, c*norm_rv.pdf(x), color = 'b', lw = 5, label = 'c*g(x)')"},{"cell_type":"code","execution_count":19,"metadata":{"collapsed":false,"id":"2ECA7B4FFF814FB0A66A343AA750A83A","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"output_type":"execute_result","data":{"text/plain":"<matplotlib.lines.Line2D at 0x7f7482adcb50>"},"metadata":{},"transient":{},"execution_count":19},{"output_type":"display_data","data":{"text/plain":"<Figure size 432x288 with 1 Axes>","text/html":"<img src=\"https://cdn.kesci.com/upload/rt/2ECA7B4FFF814FB0A66A343AA750A83A/rjw4qcw2w1.png\">"},"metadata":{},"transient":{}}],"source":"x = np.arange(-4, 6, 0.01)\nplt.plot(x, p_func(x), color = 'r', lw = 5, label = 'p(x)')\nplt.plot(x, c*norm_rv.pdf(x), color = 'b', lw = 5, label = 'c*g(x)')\nplt.gca().axes.set_ylim(0, 1)\nplt.axvline(x=0, ymin = 0, ymax=0.4,color='grey')"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"C834381B61DB44F39FC5DDDF0FA9BA03","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"蒙特卡洛的步骤：\n\n* Step 1: sample from $Q$,  get a $Y$;\n\n* Step 2: sample from $uniform(0,1)$, get a $U$; \n\n* Step 3: if $\\frac{p(Y)}{ C q(Y)} \\ge U$ , accept; else refuse."},{"cell_type":"code","source":"uniform_rv = uniform(loc= 0, scale = 1)  # 定义从均匀分布中进行取值的函数","metadata":{"id":"6ECFAE6FD84640BF8BAAD670ED7CE28B","notebookId":"634d0eaaa57b37b7a0bfc55b","jupyter":{},"collapsed":false,"scrolled":false,"tags":[],"slideshow":{"slide_type":"slide"},"trusted":true},"outputs":[],"execution_count":13},{"cell_type":"code","execution_count":14,"metadata":{"collapsed":false,"id":"D5498E6F66EB482C8B3760417750E33B","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[],"source":"sample = []\nfor i in range(100000):\n# step 1 从平均值为0，标准差为1的正态分布中采样一个值，记为Y\n    Y = norm_rv.rvs(1)[0]\n# step 2 从0到1的均匀分布中采样一个值，记为U\n    U = uniform_rv.rvs(1)[0]\n# step 3 如果$\\frac{p(Y)}{ C g(Y)} \\ge U$，则接受，否则拒绝\n    if p_func(Y) >= U * c * norm_rv.pdf(Y):\n        sample.append(Y)    "},{"cell_type":"code","source":"Y0 = norm_rv.rvs(1)[0]\nY0","metadata":{"id":"D3CE51823B1F4B9C8F9C515C362E0FE9","notebookId":"634d0eaaa57b37b7a0bfc55b","jupyter":{},"collapsed":false,"scrolled":true,"tags":[],"slideshow":{"slide_type":"slide"},"trusted":true},"outputs":[{"output_type":"execute_result","data":{"text/plain":"1.207452233772809"},"metadata":{},"transient":{},"execution_count":25}],"execution_count":25},{"cell_type":"code","source":"U0 = uniform_rv.rvs(1)[0]\nU0","metadata":{"id":"4D4FA44E78ED4EC8960D0F0B1826633A","notebookId":"634d0eaaa57b37b7a0bfc55b","jupyter":{},"collapsed":false,"scrolled":false,"tags":[],"slideshow":{"slide_type":"slide"},"trusted":true},"outputs":[{"output_type":"execute_result","data":{"text/plain":"0.654980833436986"},"metadata":{},"transient":{},"execution_count":26}],"execution_count":26},{"cell_type":"code","source":"p_func(Y0)","metadata":{"id":"ED5285D4335F44FD8912595C36742E15","notebookId":"634d0eaaa57b37b7a0bfc55b","jupyter":{},"collapsed":false,"scrolled":false,"tags":[],"slideshow":{"slide_type":"slide"},"trusted":true},"outputs":[{"output_type":"execute_result","data":{"text/plain":"0.18010477905552771"},"metadata":{},"transient":{},"execution_count":27}],"execution_count":27},{"cell_type":"code","source":"c * norm_rv.pdf(Y0)","metadata":{"id":"8583AC4ACB72406C80CE285D2F0BE015","notebookId":"634d0eaaa57b37b7a0bfc55b","jupyter":{},"collapsed":false,"scrolled":false,"tags":[],"slideshow":{"slide_type":"slide"},"trusted":true},"outputs":[{"output_type":"execute_result","data":{"text/plain":"0.9845988880399426"},"metadata":{},"transient":{},"execution_count":28}],"execution_count":28},{"cell_type":"code","source":"U0 * c * norm_rv.pdf(Y0)","metadata":{"id":"382CB2C6A15145609254C912592D3F10","notebookId":"634d0eaaa57b37b7a0bfc55b","jupyter":{},"collapsed":false,"scrolled":true,"tags":[],"slideshow":{"slide_type":"slide"},"trusted":true},"outputs":[{"output_type":"execute_result","data":{"text/plain":"0.6448934002895312"},"metadata":{},"transient":{},"execution_count":29}],"execution_count":29},{"cell_type":"code","source":"p_func(Y0) >= U0 * c * norm_rv.pdf(Y0)","metadata":{"id":"5D776B8012EA4C80963BF660E81659FD","notebookId":"634d0eaaa57b37b7a0bfc55b","jupyter":{},"collapsed":false,"scrolled":false,"tags":[],"slideshow":{"slide_type":"slide"},"trusted":true},"outputs":[{"output_type":"execute_result","data":{"text/plain":"False"},"metadata":{},"transient":{},"execution_count":30}],"execution_count":30},{"cell_type":"code","execution_count":15,"metadata":{"collapsed":false,"id":"5F97F1306B5641A8AE4ECBB875F8B7CC","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAXQAAAD3CAYAAAANMK+RAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAmdElEQVR4nO3de1yUZcL/8c/AAJm4meUjZYGthpZmxPZspyVtiTzsIfMQpGJ22rLHNp9MW2t/Zj5Guv2s1na102uptKfQMnPbrTZayyI1Q9EAQdEkj6DhaVAZhrmfPwZGUWQQZ+a+Z/i+Xy9fMMzNzNeSr5fXfd3XbTMMw0BEREJehNkBRETEP1ToIiJhQoUuIhImVOgiImFChS4iEibswX7Da665hq5duwb7bUVEQtqOHTtYtWpVs8cEvdC7du3K4sWLg/22IiIhbejQoT6P0ZSLiEiYUKGLiIQJFbqISJhQoYuIhAkVuohImFChi4iECRW6iEiYUKGLiIQJFbqISJhQoYuEiLi4bthsNuLiupkdRSwq6Jf+i0jrVFSUAwYVFTazo4hFaYQuEgI0KpeWUKGLhADP6FykeSp0EZEwoUIXEQkTKnQRkTDhc5WL2+1m2rRplJaWEh0dzYwZM0hISPA+/8UXX/DXv/4VwzDo3bs3Tz75JDabzsKLBE4McXHd2L17q9lBxGJ8jtBzc3NxOp3k5OQwceJEZs6c6X3O4XDw7LPP8tJLL7Fo0SK6du3Kvn37AhpYRGp0klSa5LPQ8/PzSUlJASApKYnCwkLvc2vXriUxMZFZs2YxcuRIzj//fDp16hS4tCIicko+p1wcDgexsbHex5GRkbhcLux2O/v27WPVqlUsWbKEs88+m1GjRpGUlMQll1wS0NAiInIynyP02NhYqqurvY/dbjd2u+fvgY4dO3LFFVfQuXNn2rdvz9VXX82GDRsCl1akDbsceIY/8BAQbXYYsSSfhZ6cnMzy5csBKCgoIDEx0ftc79692bhxI1VVVbhcLtatW0ePHj0Cl1akjRoMfAtMZDZzgDwAh8PUTGI9Pqdc0tLSyMvLIyMjA8MwyMrKIjs7m/j4eFJTU5k4cSL33nsvAAMHDmxU+CLiB9u2sQDYAAxkB7+gC4sAHnwQ3nzT3GxiKTbDMIxgvuHQoUNZvHhxMN9SJLSNGcOR+fPpA2zBAGz8D/BHYFCnOD6u2k2XLglaxhjmWtKdurBIxMo2boQFC3gR2HLcl2cBdO7Mg1W78ezAqGWMokIXsbZ588BuZ/YJX3YA3H8/vwK68X3wc4klqdBFLCgurhsxNhv7/zwHhg6lsqmD7r8fA7iL7CCnE6tSoYtYUEVFOWn8nY6Gm0E5OU0fdNFFfA6kc4rnpc1RoYtY1DDeYz/wGTWnPCYH6MlG+gYrlFiaCl3EguzArXzA34HaZi4j+qD+4+BghBLLU6GLWFA/oBP7eM/HcZXAWpIYEIRMYn0qdBELSgOcRJHbgmM/ZiA3AD3+Iz7AqcTqVOgiFpQKrOA6qk95RIz3vgOfMIAooPeeXdhsNt1Qug1ToYtYzY8/kgzkcnMzB9UAnou8v+Z6qoFUXOgio7ZNhS5iNcuWEYGvQj+mlmhWAjcENJSEAhW6iNUsW8YhYDX/2eJvyQOSgFgOBSiUhAIVuojVrFjBKqDO92aoXnlAJHANqwKVSkKACl3ESqqrYf16Vpzmt60E3MANnp3SpY1SoYtYyerVUFd32oV+EFiPCr2tU6GLWMkKT5WvbMW3fgP8jHwgWksX2ygVuoiVrFjBpkg7+1rxrWuA86giHqeWLrZRKnQRqzAMWLGCr+pcrfr2NfUff+a/RBJiVOgiVrF5M+zde9rz5w2+A1xEkuzPTBJSVOgiVvHtt4BnLrw1jgJF9Faht2EqdBGrWLsWoqMpPoOXWEOyplzaMBW6iEV88cIc8p1Oas/gNdaQTBfgQn+FkpCiQhexAsOgt/Moa7nnjF5mTf2Ei6Zd2iYVuogVbNvG+Rwr5NZax5UAXOWHSBJ6VOgiVrB2refDGVZxNbFsBnr7IZKEHp+7/7jdbqZNm0ZpaSnR0dHMmDGDhIQE7/MzZsxgzZo1tG/fHoC5c+fSoUOHwCUWCUdr1lAHrPfD7Z6LUKG3VT4LPTc3F6fTSU5ODgUFBcycOZN58+Z5ny8qKuK1116jU6dOAQ0qEtbWrqUUOEz7M36pQmAQgNMJ0ae+wbSEH59TLvn5+aSkpACQlJREYWGh9zm32015eTlTp04lIyODd999N3BJRcJZQQFr/fRShUAUwMaNfnpFCRU+R+gOh4PY2Fjv48jISFwuF3a7ncOHDzN69Gjuuusu6urqGDNmDH369KFXr14BDS0SVg4cgG3b+M5PL+cdchUWQp8+fnpVCQU+R+ixsbFUVx+7Va3b7cZu9/w90K5dO8aMGUO7du2IjY3l2muvpaSkJHBpRcLRhg0AZ3RB0fFKARd4Cl3aFJ+FnpyczPLlywEoKCggMTHR+9zWrVu54447qKuro7a2ljVr1tC7t07HiJyWoiLPBz+9nBPYdNzrStvhc8olLS2NvLw8MjIyMAyDrKwssrOziY+PJzU1lVtvvZXbb7+dqKgobr31Vi699NJg5BYJH8XFcNZZbD161G8vWQhcphF6m2MzDMMI5hsOHTqUxYsXB/MtRaxt4ECorMS2di1gALb6Jxo+b+przT8/FXjKZgOHA84+Owi/CQm0lnSnLiwSMVtxMe9u8O+KlGLw7K+ulS5tigpdxEwHD8K2beQfrfZ97GnwLk3QIoU2RYUuYiY/r3BpUAZgs0FpqZ9fWaxMhS5iJj+vcGlwFCAhQSP0NkaFLmKm+hUu3/v9hWP4aOtW1r+rBQhtiQpdxExFRdCrF26/v3ANpTxMd5cT3P5/dbEmFbqImYqL4fLLA/LSJfSiPXBxZCRxcd0C8h5iLSp0EbMcOgQ//AABurq6lJ4A9ORTKirKA/IeYi0qdBGz1K9wCdQIvaHQe6ETo22FCl3ELA17rQRohL6LCzgI9ERLF9sKFbqIWYqLISYGfvrTAL2BjVJU6G2JCl3ELPUrXIiMDNhblKApl7ZEhS5iluJiFpduwmaz+T62lUqBeLah7bnaBp/b54pIADgcUF5OPtB4t0T/aphsSWz2KAkXGqGLmCFAe7icqGGvxR4Bfh+xBhW6iBkCtIfLiTbXf1Shtw0qdBEzFBdDdDRbAvw21cAu4tB9xNoGFbqIGepXuNQF4a02cSk9sOny/zZAhS5ihgDu4XKiMnrQA0OX/7cBKnSRYHM4YOvWgF0heqIyenAh0D4o7yZmUqGLBFvDTSeCNELfVD+D3j0o7yZmUqGLBFuA93A5UVn9GhetdAl/KnSRYKtf4UL34IyZVehthwpdJNiKiih2G9iiooLydg46sBu0dLENUKGLBFtxMetdtXgu+Q+OMjRCbwt8Frrb7Wbq1Kmkp6eTmZlJefnJS5/cbjf33nsvb7/9dkBCioSN6mr4/vuAXyF6IhV62+Cz0HNzc3E6neTk5DBx4kRmzpx50jEvvPACBw8eDEhAkbBSv8Il0Hu4nGgTcBHA4cNBfmcJJp+Fnp+fT0pKCgBJSUkUFhY2ev7jjz/GZrN5jxGRZgRpD5cTlTV8snlzc4dJiPNZ6A6Hg9jYWO/jyMhIXC4XABs3buTDDz/k4YcfDlxCkXBSXIyTY5tmBYu30MvKmjtMQpzP/dBjY2Oprq72Pna73djtnm9bsmQJFRUV3HnnnezYsYOoqCi6du3KjTfeGLjEIqGsqIiNgCvIb+ut8U2bgvzOEkw+Cz05OZlly5YxePBgCgoKSEw8tlX+5MmTvZ+/+OKLnH/++SpzkeYUFwd9ugXgIFAJ/IdG6GHNZ6GnpaWRl5dHRkYGhmGQlZVFdnY28fHxpKamBiOjSHg4fBi+/z7oJ0QblKFCD3c+Cz0iIoLp06c3+lr3Jq5we+ihh/yXSiQclZSAYZgyQgfPSpfrNeUS1nRhkUiw1K9wMXOEzvbtcOSISQkk0FToIsFSXAx2O2aNkb2TLVsCfZ8kMYsKXSRYioogMTHoK1waeP8i0bRL2FKhiwRLcXHQtsxtymaiAXgq8y7TMkhgqdBFguHIEc9UR5BuatGU/TjZy3nEOfablkECS4UuEgwbNoBhQJ8+psYoo4fuXBTGVOgiwRDkuxSdyiYu1b7oYUyFLhIMhYUQFQU9zN3EtoweXAxw9KipOSQwVOgiwVBUBL16eUrdRGX08PzQf/+9qTkkMFToIsFQVGT6dAscu7+odl0MTyp0kUBzOGDrVksU+qaGGXStRQ9LKnSRABuU4CnRsf9/jslJYB+dqAJe/+OTZkeRAFChiwRYXNVuAPIO7DE5iUcZcOERh9kxJABU6CIB1gc4wllsIRqbzWZ2HN0wOoyp0EUCrDewgctw4wQMs+NQBiQAOJ0mJxF/U6GLBFhvoAjzT4g22AREgpYuhiEVukgg7d/PxUAh5l7yfzzdMDp8qdBFAujXP+0FWGuErkIPXyp0kQC6YF8FYK1C3wscAF6cMIG4uG4mpxF/UqGLBNAVgAMo95yGtIxNwKUMoKKi3Owo4kcqdJEAuhJYDxgW+1HzLF3UlEu4sdafMpFwYhhcCRSYnaMJZUA3tmI3O4j4lQpdJFDKy+kIrDM7RxPKADt1FpsIkjOlQhcJlHWeKrdioTdszaWbXYQXFbpIoBQU4Aa+MztHExpmz7UFQHjxWehut5upU6eSnp5OZmYm5eWNz4q/9dZbDBs2jOHDh/PPf/4zYEFFQs66dWwCDpudowmVwCFiVehhxuc5kdzcXJxOJzk5ORQUFDBz5kzmzZsHQFVVFW+//Tbvv/8+NTU1/OpXv2LQoEGW2IBIxHTr1lnyhGiDMnrQw9IJ5XT5HKHn5+eTkpICQFJSEoWFhd7nOnXqxJIlS4iKimLv3r3ExMSozEUADh6ELVssOX/eoIwemkMPMz4L3eFwEBsb630cGRmJy+XyPrbb7SxYsID09HR++9vfBialSKhZvx6w5gnRBpu4lEsAjvt5ltDms9BjY2Oprq72Pna73djtjWdqRo8ezZdffsnq1atZuXKl/1OKhJr6FS4F5qZoVhk9iAL44Qezo4if+Cz05ORkli9fDkBBQQGJiYne57Zs2cL48eMxDIOoqCiio6OJiNDCGZEFk/7AXmCn2UGaoRtGhx+fJ0XT0tLIy8sjIyMDwzDIysoiOzub+Ph4UlNT6dWrF+np6dhsNlJSUvj5z38ejNwiltb7iIN8bgH+ZXaUU2pU6LfcYm4Y8QufhR4REcH06dMbfa179+7ez8ePH8/48eP9n0wkVB05whXATP4TKxf6Li6gGmi/aZPPYyU0aH5ExN/WrcMOfMvVZifxwea5wEhTLmFDhS7ib6tXez7wnyYH8a2MCDZ8+KH2RQ8TKnQRf/v2W3YBO7nQ7CQ+leHmp0SzR/uihwUVuoi/rV6NZ4xu/YvsNgExOLnY7CDiFyp0EX86dAhKSvjW7BwtpE26wosKXcSf1qwBw1ChiylU6CL+9PXXAKwyOUZL7QSOcJb2dAkTKnQRf8rLoxioMjtHCxnARhLpaXYQ8QsVuoi/uN3w9dfkmZ3jNJXSk57YtHQxDKjQRfylpAT27Qu5Qi+hFz/FYJ+WLoY83fRbrGP/figogN274ehR6NwZuneHnj0hFPbZz/NUeSgWeiQ6MRoOVOhiLocD3ngD3noLVqxo+pi4OBgyBB54AK68MqjxTkteHnTuTNmePWYnOS2l9TPomkcPfZpyEXO4XPD883DJJTB+PBw+DE89BR99BMXFsHmzp+Bfew369YPXX4ekJLj1Vs/UhhXl5cENN5id4rQ1FHovk3PImdMIXYKu3/kXMufAj1zpckJamqfIr7uu0TFxcd2oqCinS5cEdu/eClVVMG8ezJoFffsyI6Y9Tzr207lLAkDjY82wbZtnk6sHH4QlS8zJ0ErVxLINFXo40AhdgmvRIv7x4y7iXE5YtAg++cRb5nFx3bDZbN4yB6P+I9CpEzzxhKc0hwzhj4795NIfV0X5yceaITcXgCseecS8DGegFE25hAMVugSHYXhG4rffznrgKoDhw4m74BLvcjlfxRwX1w1bly7EfbGKO4HrWME3wOVB+Q34kJvLbqAQt9lJWqWE+hG6YZicRM6ECl0CzzBg0iSYNg3uvJObgF31T1V4R9jHi2nyZbyFX/kDbwL9+Zyzga+Aq+u3wzKFYUBuLp4xegisxmlCCXAOeFYYSchSoUtgGQZMmACzZ3tOfv7tbzhPOigGW6NliTWNnjvVBS+ruJZrgX3AZ6Ryvf9Sn57CQqisrC/00OQ9zWzVE87SIip0CaxnnoE5czylPmcONHkT8Ro8F6E3pabZufFy4EY8t1P7GOBbE7bF+vRTAD4L/jv7Tan3k9LmDhOLU6GL38XFdfOMqrOzPScyR4/2jNAbjcJPHJU358RjG0/J7ABuYhl7AQYNCmopxcV144spT0CvXmwP2rv63w7AARqhhzgVuvhdRUU53SvK4f77+Tz6LC7+13KIiPCuYvFoblR+ohOPbTwlA7CLC7mFKCr37mX75b1hx44z/n20xNGKcq53HoXf/jYo7xcoBvWjdBV6SFOhi99dALwLkJDAEOdRtlf+ABw7qelfx8q9jFoGks857jpPwVZX+/m9TjYIiALPlawhrhQ05RLiVOjiXzU1vAt0AHj/fQ4Apze9cmbWkkwGdtxr1sCdd4Lb3Wh9u7/dCuwGIq437ZSs35QAlJd7rtqVkKRCF/96/HGuB+4C6NOn/ounM71y5v6Ji0cB3nsPnnwycBceHT7Mr4ClgBHE31+gFINnVZJG6SHLZ6G73W6mTp1Keno6mZmZlJc3/qF4/fXXGTFiBCNGjOAvf/lLwIJKCPjXv+C55/gr8G4QR+VNeR7gnntgxgzuaOY47wnc1li6lA7AW637bssp9EweMb5/mslJpLV8Fnpubi5Op5OcnBwmTpzIzJkzvc9t27aNpUuX8s4777Bw4UK++uorSnRSpW3as8czxXH55Z7RcZBH5U2aOxduvJG/AdewsslDmr6wqYUWLOAH4MtWB7SWMmqpIZqLD/5odhRpJZ+Fnp+fT0pKCgBJSUkUFhZ6n4uLi+O1114jMjISm82Gy+UiJqbpq/wkjBmGZzRcVQVvv81Rs/MAEIMtJobLN2xhB7CEIVzsz5evqICPP+YtTP9ry2/q8OyN3ocI3b0oRPksdIfDQWxsrPdxZGQkLpcLgKioKDp16oRhGMyaNYvLL7+cSy65JHBpxZqys+Hvf/fuhGgNnn8hbNiznd8A7TjCUvDsv+4Pr7wCdXVk++fVLKOQPvTBbe5GZ9JqPgs9NjaW6uOWf7ndbuz2Y7vu1tTU8Oijj1JdXc2TTz4ZmJRiXTt3cuDee/kCuOCZ581O06QNwO0s5AqAUaM89/5shYbVMhd1SfBs5TtwIJv8GdQCCulDAvWrlCTk+Cz05ORkli9fDkBBQQGJiYne5wzD4MEHH6Rnz55Mnz6dyMjIwCUV6zEMPrmkB1GGwT1sYnf9enNr8UwB/osBTABYupQXO5zbqimFhtUy/Sp/gF274KGH/JjTGgrxrEzqbXIOaR2fN7hIS0sjLy+PjIwMDMMgKyuL7Oxs4uPjcbvdfPPNNzidTr780nNq6JFHHuGqq64KeHCxgHfeYYDzCBOBzfQgmOvNW+7YhUd/AV4cN46H5s1jzeGDrXo1O7VMA7jiChg40B8BLaWh0Pv4OE6syWehR0REMH369EZf6969u/fz7777zv+pxNLi4rpxvruOQvcRVgEveJ9pWNlitVJvEEPUvHn8E3gVWnVnobvI5lKAp58+xUZjoa2cBByo0ENV+P2JlICrqChnyp7tcOgQd0MI3dKhBhcGQ4FvAdLTOZ0V112AZ5jCVwC//nUA8pnPIIIiVOihSoUup+0mYBTAY495ri4MMQ48e7Bw2WUsBYa15Jvcbl4G2lPNvXDCzpHhpRDNoYcqFbqcHqeTvwKbAaZMMTlM6+0nhvPWrSMfWAiQlQV1dU0fbBgweTK3An9gJuF+YXwREAeei8UkpKjQpcXi4rrxh5gYLgMeAmjXzuREZ6KGKgxuBhYSAU88wYp27SE/v/FhBw7AfffB7Nm8CPyZh80IG1Tes2Lr1pkZQ1pBhS4tFlNRzlTasRj4yOwwfnIUuAM3Y3iDnrU1cPXVFEZFs7BdLItsNg6eey787W/w+OP1Ve6Zamm8t3t4Wev9ZG1zh4kFqdClxf4MGNg867ktuUSx9eYzhm7AI8xmj8vFz49Wk0QPFhs2riYGW1ZWo0v8A7O3uzX8CPwAKvQQpEKXlvn73xkCPMWTbAMssfmWnx0CnucRfonBJUAim7gLN/nGURr/XsN/v6I1oEIPQSp08e3wYfj97ykCXqgfn7dtNb4PCXFrwbMvur/2vpGgUKGLb888A1u38iBQS7TZaSQI1mIHw+DX8T3MjiKnQYUuzSsthT/9CTIzWW52FgmaNXh2VO22r8LkJHI6VOjSpIZVHF/0TeJAbS1d5s83O5IE0Q5gD+ejXZlCiwpdmlRRUc7tvEM/51GmGAaVYXYCVHxbQ7IKPcSo0KVJHYDn+W/ysfGy2WHEFGu5yrOni9NpdhRpIRW6NGkaEMduxmGE0OZb4k9rSPacAj/utpNibSp0Odn69fweeIXfsdrsLGKab7na88lq/SkIFSp0aczthnHj2Ac8TpbZacRE33MJlQArV5odRVpIhS6NvfEGfP01k4B9dDI7jZjKxkpQoYcQFbocU1UFkyfzFfCm2VnEElYClJTAvn1mR5EWUKGL1xsXJeDau5cHCbddWqS1vGPzb74xM4a0kApdPL75hswjDubw3+gusdJgNYDNxrPD0omL62ZyGvFFhS6eO/WMG8cuYJrnnvYigOd2ffTpQ5/qA/VbBouVqdCF/9exM6xZwyPAIX5idhyxmmuv5Voabu0hVqZCb+u2buUPjn18yK8899YUaSSGMa++yrnguWpULE2F3pYZBjzwAG7gQeaanUYsqYYv2ApAf1NzSEuo0Nuyt96CTz5hCrCNeLPTiEX9QALfA/3MDiI++Sx0t9vN1KlTSU9PJzMzk/Lyk0+MVFVVMWDAAGpqwv9OLmFjzx6YMIGvgXlmZxHL+5z6QndrZx8r81noubm5OJ1OcnJymDhxIjNnzmz0/Jdffsndd9/Nnj17AhZSAmDCBDh4kPtAm2+JT18A5wMUF5ucRJrjs9Dz8/NJSUkBICkpicITdl6LiIggOzubjh07BiSg+N9dHf8D/vd/mVZbi348pSU+937y+akPEtP5LHSHw0FsbKz3cWRkJC6Xy/v4hhtu4Nxzzw1MOvG/ykpmHdhDPvA02udaWqYcPKdG//1vc4NIs3wWemxsLNXV1d7Hbrcbu90e0FASIIYBv/sdPwHGAC6izE4kIeQTgNxc3fDCwnwWenJyMsuXe24PXFBQQGJiYsBDSYC8+SZ88AFPgKZa5LR9hB0OHWJonFZEWZXPoXZaWhp5eXlkZGRgGAZZWVlkZ2cTHx9PampqMDKKP5SXc/CuuygAXjA5ioSmz3DhJIpr9u0lLq4bu3dvNTuSnMBnoUdERDB9+vRGX+vevftJx/1bc2uWdVGXBBZV/kBvYCybcXPy/z8RXxzAV/yCQSzjD9rXxZJ0YVEb8FDlD1wH3At8z0/NjiMh7CMG0RfoanYQaZIKPcyNPLcLjwEvcT+LzA4jIe+fDAbgNybnkKap0MPZtm38eX8l64H/5nmz00gYKOZyNgAjzA4iTVKhh6luXeL5Nj6eGOB24CjtzI4kYcHGIuq3AaisNDmLnEiFHo4Mg/+p3EYyNkYBpWbnkbCyCIgEWLzY5CRyIhV6OHruOTKBqUznQ7OzSNgpBEoAFmoHfatRoYebpUth8mTeBZ7mCbPTSJjKIZK6ZctI7nyR2VHkOCr0cPL115CeDldfzZ2AbhomgfI6dUQCv967w+wochwVerjYsAF+8xu4+GL48EMOm51HwtpW4FNu5h7w3GRcLEGFHg62boWBAyEqCj7+GDp3NjuRtAGvch8J4NmwSyxBhR7qvv8e+vWDQ4e4uTaCuOt/aXYiaSM+4Fb2AMzV/WitQoUeyrZsgf794dAhyM3ls6pdVFTsxmbT3LkEnpMYz63Fly7VnYwsQoUeQuLiuhEX183zoLDQMzJ3OOCzzyA5uf6oGsAwKaG0NS8CnH02/OlPZkcRVOghpaKinIqKcs9twH7xC6ir45fEEHn1LzQqF1P8SAwvHD5M7RtvQBM3kJfgUqGHmEzs1Nx0E6WOw7ByJcuqduF2H0ajcjFHDbP5gTpgUa/eZodp81ToocLp5C/Am7j4il9ybV0txOvOMWK+7VzMc8CIo9XwzTdmx2nTVOihYPNm6NeP/wL+BAzgE/YTo2kWsYxngN0AEyaA221umDZMhW5lhgEvv0z1pZeyf+VKhgOPAXXY0clPsRIHMBlgxQp4Xls1m0WFblVFRZCaCg88wNeGQR+28Z7ZmUSaMZ9o3gdqHn2Um8670Ow4bZIK3Wr27YNHHoErr4SCAu4HBgA70CZIYnVOfkcl+4BXqnbBXs/NpG0227HlthJQPm8SLUGyfz+88AIHpk+ng2HwKnae2LePH83OJXIa9tKZocC/gbzOnTkAgEFFhc73BING6GYrK4OJEyEhAZ56ilzD4CoKeAAXP2qOXELQCmA0cB02PgJ+Ul/rEngq9CCLi+tGB5uNced0hkGD4NJLqX3uOd45eJBfdrqA4cB6rjQ7psgZeQ8YzQJuAJZzIz3MDtRGqNCDZccOePNN5laUs4cY5h3cC4WF/Kn9OcQDd2CwrKrK7JQifvM2I/k1cBHbWQueTbxcLpNThTcVeiDU1Xk2K3rzTRg/nlJ7NFx0Edx5J9cBr/A7biCKiO3beaz6gGf9LuBZiigSPv4FXMk6VmKD//oviqKieOic8+HoUbOjhSWfJ0XdbjfTpk2jtLSU6OhoZsyYQUJCgvf5hQsX8s4772C32xk3bhw33XRTQANbRl0d7N0L27fDpk2Nf333HVRXA1Bts7HVMHiVZ8nlcdZTi8EcPNsaGeiuQhLudnARaRgMAZ7mMl48uMEzwLntNhg8GG68Ec47z+yYYcFnoefm5uJ0OsnJyaGgoICZM2cyb948APbs2cP8+fN57733qKmpYeTIkdxwww1ER0f7L6FheH4d/3lrfoHnn3tOJ9TUNP544tcOHYKDB+HAgUYfcxctprO7jis7nweVlSfdqWVHRCRb7FEUOI+yGviWIkqN3nium3sUmOS//y4iIWYJsIQibiKG+3/cz8DXXuOc117zPJmQ4Fmqm5Dg2dLiwguhY0c45xzPxw4dIDracxOX439FRoKumPbyWej5+fmkpKQAkJSURGFhofe59evXc9VVVxEdHU10dDTx8fGUlJTQt2/fU7/g2rWe7TZbWsImqwUO1P86F9hOBN/u2sVu7OwCdtuiKTWcbKaaI+5O4DzKsZH35UAMmkoRaWBjGbUsA+w4uZ72/Jxafla+nSt37OSydmd5BlSn4/hSP/FjU19ryTFW07kzJCX5PMxmGM035xNPPMEtt9xCv379AOjfvz+5ubnY7XY++OADNm7cyKRJnpHn5MmTGTJkCNdff/0pX++aa66ha9eup/E7ERGRHTt2sGrVqmaP8TlCj42Npbp+Phg8c+p2u73J56qrq+nQoUOzr+crkIiItI7PVS7JycksX74cgIKCAhITE73P9e3bl/z8fGpqajh06BCbN29u9LyIiASPzymXhlUuGzduxDAMsrKyWL58OfHx8aSmprJw4UJycnIwDIP777+fAQMGBCu7iIgcx2ehi4hIaNCFRSIiYUKFLiISJlToIiJhIuiFfvjwYcaNG8eoUaMYO3YsFRUVwY7QIocOHeKBBx5g9OjRpKens3btWrMjNevTTz9l4sSJZsdoxO12M3XqVNLT08nMzKS8vNzsSM1at24dmZmZZsc4pdraWiZNmsTIkSMZPnw4n332mdmRmlRXV8eUKVPIyMjgjjvuYOPGjWZHOqUff/yRfv36sXnzZrOjNOu2224jMzOTzMxMpkyZcsrjgn6Di4ULF9K7d2/Gjx/P4sWLefXVV/njH/8Y7Bg+ZWdnc+211zJ27Fi2bNnCxIkTef/9982O1aQZM2bw1Vdfcdlll5kdpZHmto2wmldffZWlS5fSrl07s6Oc0tKlS+nYsSPPPvss+/fvZ8iQIaSmppod6yTLli0D4J133mHVqlU8//zzlvz/Xltby9SpUznrrLPMjtKsmpoaDMNg/vz5Po8NeqGPHTuWuvo9UHbu3MlPfvKTYEdokbFjx3r3pKmrqyMmJsbkRKeWnJzMzTffTE5OjtlRGmlu2wiriY+P58UXX2Ty5MlmRzmlgQMHepcFG4ZBZGSkyYmadvPNN9O/f3/A2j/js2bNIiMjg1deecXsKM0qKSnhyJEj3H333bhcLh555BGSTrENQEALfdGiRbzxxhuNvpaVlUXfvn0ZM2YMGzduJDs7O5ARWqS5nHv27GHSpEk8/vjjJqU75lQ5Bw8ebMkrcB0OB7Gxsd7HkZGRuFwu75XGVjJgwAC2b99udoxmtW/fHvD8d/3973/PhAkTzA3UDLvdzmOPPcann37KnDlzzI5zksWLF9OpUydSUlIsX+hnnXUW99xzDyNGjGDr1q3cd999fPzxx03/HBkmKisrM1JTU82M0KySkhJj8ODBxueff252FJ9WrlxpTJgwwewYjWRlZRn/+Mc/vI9TUlJMTOPbtm3bjBEjRpgdo1k7d+40brvtNmPRokVmR2mRyspKo3///kZ1dbXZURoZOXKkMWrUKGP06NHGz372M2PYsGFGZWWl2bGaVFNTYxw5csT7eNiwYcbOnTubPDboJ0VffvlllixZAnhGHFb9Z2NZWRkPP/wws2fP9m5MJqenuW0j5PTt3buXu+++m0mTJjF8+HCz45zSkiVLePnllwFo164dNpuNiAhrLah76623WLBgAfPnz+eyyy5j1qxZdO7c2exYTXr33XeZOXMmABUVFTgcjlNmDfq/fYcNG8Zjjz3Ge++9R11dHVlZWcGO0CKzZ8/G6XTy9NNPA56NyKx4YsfK0tLSyMvLIyMjw7tthLTeSy+9xMGDB5k7dy5z584FPCdzrXZS75ZbbmHKlCmMGjUKl8vF448/brmMoWT48OFMmTKFO+64A5vNRlZW1imnLXXpv4hImLDWv4NERKTVVOgiImFChS4iEiZU6CIiYUKFLiISJlToIiJhQoUuIhIm/g97P5e1tFsKrgAAAABJRU5ErkJggg==","text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":"x = np.arange(-3, 5, 0.01) # \n\nplt.gca().axes.set_xlim(-3, 5)\n\nplt.plot(x, p(x), color = 'r')\n\nplt.hist(sample, color = 'b', bins = 150, density = True, edgecolor = 'k')\n\nplt.show()"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"45B9F4373D8747488CAAB72D60C8531D","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"使用接受-拒绝采样，我们可以解决一些概率分布不是常见的分布的时候，得到其采样集并用蒙特卡罗方法求和的目的。但是接受-拒绝采样也只能部分满足我们的需求，在很多时候我们还是很难得到我们的概率分布的样本集。比如：\n\n- 对于一些二维分布p(x,y)，有时候我们只能得到条件分布p(x|y)和p(y|x)和,却很难得到二维分布p(x,y)一般形式，这时我们无法用接受-拒绝采样得到其样本集。\n\n- 对于一些高维的复杂非常见分布p(x1,x2,...,xn)，我们要找到一个合适的q(x)和c非常困难。"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"5142B914B2174B9A8873EF3D109B3AA9","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"\n从上面可以看出，要想将蒙特卡罗方法作为一个通用的采样模拟求和的方法，必须解决如何方便得到各种复杂概率分布的对应的采样样本集的问题。这就需要用到马尔科夫链。"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"0CAAB8541B5642558BB3848842B62B03","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"#### 马尔科夫链"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"099613496A9A4E029D29D68921B458C2","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"马尔科夫链假设某一时刻状态转移的概率只依赖于它的前一个状态。\n\n$P(X_{t+1}|...X_{t−2},X_{t−1},X_{t})=P(X_{t+1}|X_{t})$\n\n即，某一时间点的状态$x_n$在其下一个状态$x_{n+1}$的概率，仅仅取决于现在它所处的状态，而与它如何达到现在的状态无关。\n\n换句话说，该随机过程没有记忆。"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"E4F55F31553146819FCFAA00F67012AF","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"考虑以下这样的情况，我们在抛地球仪时，手指会指向陆地或者海洋，每一次抛掷就是一次状态转移。\n\n状态转移的情况有以下四种：\n\n- 陆地 -> 陆地\n- 陆地 -> 海洋\n- 海洋 -> 陆地\n- 海洋 -> 海洋"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"8EF534B9767144C39AB4DE02A72D6403","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"状态转移矩阵为\n\n$\\begin{bmatrix}\n     &  陆地 & 海洋\\\\\n陆地& 0.75 & 0.25\\\\\n海洋 & 0.1 & 0.9\\\\ \n\\end{bmatrix}$"},{"cell_type":"code","execution_count":22,"metadata":{"collapsed":false,"id":"E933136C3FBC4969843F168DF16089FA","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["[0.1 0.9]\n"]}],"source":"A = np.array([[0.75,0.25],\n             [0.1,0.9]]) # 定义一个矩阵\nprint(np.dot([0, 1], A))  # 矩阵乘法"},{"cell_type":"markdown","metadata":{"id":"291C43A63EA24659AFA30B76EAC64F23","jupyter":{},"collapsed":false,"scrolled":false,"tags":[],"slideshow":{"slide_type":"slide"},"notebookId":"634d0eaaa57b37b7a0bfc55b","trusted":true},"source":"无论初始状态如何，在经过n次状态转移过后，它处于某一个状态的概率是平稳分布的。"},{"cell_type":"code","execution_count":25,"metadata":{"collapsed":false,"id":"149C47749E6F459182DFBFA0E016DF28","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[],"source":"#\ndef get_matrix_pow(init, matrix, n):\n    '''\n    init:   初始状态\n    matrix: 状态转移矩阵\n    n:      状态转移次数\n    '''\n    if init[0]+init[1]==1:  # 确保初始状态概率之和为1\n\n        ret = init\n\n        rets=[]\n        rets= np.array(ret)\n        \n        for i in range(n):# n次状态转移过程\n            ret = np.dot(ret, matrix)# 矩阵乘法\n            rets=np.row_stack((rets,ret))# 结果合并\n        plt.plot(range(n+1),rets[:,0],color='blue',label='land') # 每一次状态转移，处于陆地的概率\n        plt.plot(range(n+1),rets[:,1],color='red',label='sea')   # 每一次状态转移，处于海洋的概率\n        plt.legend()\n        plt.show()\n        return ret\n    else:\n        raise ValueError(\"please make sure the sum of input equal to 1, and dim of it is 2\")"},{"cell_type":"code","execution_count":27,"metadata":{"collapsed":false,"id":"CC7B822620434DFE907D0D2304DB4575","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAXIAAAD3CAYAAAAALt/WAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAsf0lEQVR4nO3deVxU9f7H8dewDKCDGi65Ym64ZohWeg1bjBQ1vWEl5lKpZaualmb9QlNCvOaSZpqZmFoKGbduJi4YZVFZYpSQS4tZ0k1N9Cqgssz5/fGNYZUBnZnDzHyej8d5zMw5M8zHAd8cvue7GDRN0xBCCOG0PPQuQAghxJWRIBdCCCcnQS6EEE5OglwIIZycBLkQQjg5L0e/4Y033kiLFi0c/bZCCOHUsrKy2LNnT6XHHB7kLVq0IDEx0dFvK4QQTi0iIuKSx6RpRQghnJwEuRBCODkJciGEcHIObyMXQghbKCgo4NixY1y4cEHvUmzK19eXli1b4u3tXe3XSJALIZzSsWPH8Pf355prrsFgMOhdjk1omsapU6c4duwYbdq0qfbrpGlFCOGULly4QMOGDV0mxAEMBgMNGzas8V8ZVoPcbDYTFRXFiBEjGDNmDEePHi1zfNWqVQwbNoxRo0aRkpJSs6qFEOIKuFKIF7ucf5PVIE9OTiY/P5/4+HimTZtGbGys5dihQ4fYsmULCQkJrFmzhqVLl3L+/PkaF1Edx4+DdD8XQoiKrAZ5WloaoaGhAAQHB5ORkWE59vPPP3PDDTfg4+ODj48PrVu35tChQ3YpdMcOGD4cDh+2y5cXQogaSUxM5OWXX77s17/88ss2GxxpNchzcnIwmUyWx56enhQWFgLQsWNH9u7dS05ODqdPn+bbb7+12xl5377qNinJLl9eCCGcltVeKyaTidzcXMtjs9mMl5d6Wbt27Rg1ahQTJkygefPmXHfddVx11VV2KbRtW+jYEbZuhcmT7fIWQggntW4drFlj2685bhyMHWv9eQsXLiQjI4MzZ87QqVMn5s2bx7Jlyzh27BinTp3ijz/+YObMmYSGhrJ9+3ZWrFhBQEAABQUFtG3b1ia1Wj0jDwkJYffu3QCkp6cTFBRkOZadnU1ubi6bNm3ixRdf5L///S8dOnSwSWGVCQ+HTz+FvDy7vYUQQlRbQUEB9erVIy4ujvfee4/09HSOHz8OgNFoZPXq1Tz//POsXbuWgoICYmNjiYuL480338TX19dmdVg9Iw8LCyM1NZXIyEg0TSMmJoa4uDgCAwO57bbb+OWXXxg+fDje3t5Mnz4dT09PmxVXXng4LFkCKSkweLDd3kYI4WTGjq3e2bOtGQwGsrOzmTp1KnXq1CEvL4+CggIAOnfuDEDTpk3Jz88nOzub+vXrW1otevToYbM6rAa5h4cHc+bMKbOvXbt2lvvlj9lTv35Qp45qXpEgF0Lobc+ePbRu3ZolS5aQnZ3Nzp07KV7Pvnw3woYNG3L27Fmys7MJCAhg//79NG3a1CZ1ONXITl9fuO02dcFT08AFu5AKIZzItddeS2ZmJqNGjcJgMNCqVStOnDhR6XO9vLyIiopi/Pjx1K9f33Kt0RacKshBNa9s2aK6IXbsqHc1Qgh3FRERcck5wnv27Gm5365dO9avXw/ALbfcwi233GLzWpxuiH54uLrdulXfOoQQorZwuiBv0wY6dZL+5EIIUczpghxKuiGW6t4uhBBuyymDfNAgyM+Hjz/WuxIhhNCfUwZ5aCjUrSvNK0IIAU4a5D4+ZbshCiGEO3PKIAfVvPLrr2CnyRaFEMJpOG2QSzdEIYRQnG5AULHWraFzZ9W8MnWq3tUIIXSlw/SHR44cYebMmXh5eWE2m1m4cCHvvPMOe/fuxWw288ADDxAeHs7XX3/Nq6++iqZp5ObmsnDhwhqtx1kdTntGDqp5ZfduyMnRuxIhhLv54osv6N69O3FxcTz55JMkJydz7NgxNm7cyLp161i5ciVnz57lxx9/ZMGCBaxfv5477riDbdu22bwWpz0jB9W8snCh6oY4dKje1QghdKPD9Id33303b7zxBhMmTMDf359OnTqRmZnJmDFjACgsLCQrK4urr76al156iTp16nD8+HFCQkJsXotTB/lNN5V0Q5QgF0I40q5du+jZsydPPPEEW7ZsYdGiRfTt25e5c+diNpt57bXXaNWqFePGjWPnzp2YTCZmzJhhmR3Rlpw6yH184PbbZTZEIYTjdevWjRkzZrBixQrMZjNLly7lww8/5L777iMvL4/bb78dk8nE0KFDGTVqFH5+fjRq1OiSsyNeCacOclDNKx98AAcOQJcuelcjhHAXgYGBbNy4scy+bt26VXjezJkz7V6L1YudZrOZqKgoRowYwZgxYzh69GiZ42vWrCEiIoLhw4ezc+dOuxV6KcXdEGWUpxDCXVkN8uTkZPLz84mPj2fatGnExsZajp09e5Z169axadMm1qxZQ0xMjF2LrUxgIHTtKkEuhHBfVoM8LS2N0NBQAIKDg8nIyLAc8/Pzo3nz5pw/f57z589XWNrIUcLDVTfEc+d0eXshhE7sceFQb5fzb7Ia5Dk5OZhMJstjT09PCgsLLY+bNWvG4MGDueuuuxirx+qnqCAvKJDZEIVwJ76+vpw6dcqlwlzTNE6dOoWvr2+NXmf1YqfJZCK31MTfZrPZstbc7t27OXHiBLt27QJg/PjxhISE0L179xoVcaVuuglMJtW8MmyYQ99aCKGTli1bcuzYMU6ePKl3KTbl6+tLy5Yta/Qaq0EeEhJCSkoKgwYNIj09naCgIMux+vXr4+vri9FoxGAw4O/vz9mzZ2te+RUyGlU3xK1bpRuiEO7C29vb5kPdnZXVIA8LCyM1NZXIyEg0TSMmJoa4uDgCAwPp378/X3zxBffeey8eHh6EhITQt29fR9RdQXg4vP8+/PCDuvgphBDuwmqQe3h4MGfOnDL72rVrZ7k/adIkJk2aZPvKaqh0N0QJciGEO3HqSbNKa9UKunWTaW2FEO7HZYIc1Fn5559LN0QhhHtxqSAfNEh1Q/y7E40QQrgFlwryvn3B31+aV4QQ7sWlgtzbu+xsiEII4Q5cKshBNa8cOwaZmXpXIoQQjuFyQT5woLqV5hUhhLtwuSBv2RKuvVZmQxRCuA+XC3JQzSuffw46zBYghBAO55JBHh4OhYWQnKx3JUIIYX8uGeT/+AfUqyfNK0II9+CSQe7tDWFh0g1RCOEeXDLIQTWvZGXB/v16VyKEEPblskFe3A1RmleEEK7OZYO8RQu47joJciGE67M6H7kzCw+HBQvgf/+D+vX1rkYIGyoqUjPEFRSoLlpFRSW35e9bO2Y2l2zlH1vbbzarC1Hlb63tK30fSvZVZyv9/OL7Nd1X7EruW3tc/pivL8yYAU2bYmsuH+Sxsaob4vDhelcjXILZDLm5kJNTcbtwoex28WL19l24APn5JcFcUFDxcfnNbNb7k3Asg6HsVnpf8f2a7iv9tS/3vrXHpe/7+sLYsfoEudlsZvbs2Rw6dAij0Uh0dDStW7cG4MCBA8TExFiem56ezvLly+nXr5/NC70cffqoM/GkJAlygTqrPH0asrPh1KmKt6dPVx7QpbdSC5HXiNGo/iOX3nx8Sm79/FSfWW/vks1oLPv4Use8vNTm6VmylX5c1bHizcOjZCv/+FL7DYaK94tDs6p9lT2GimFd+pioktUgT05OJj8/n/j4eNLT04mNjWXFihUAdO7cmfXr1wOQlJREkyZNak2IQ8VuiPIz4WI0Dc6cUbOk/f67uv3jj0sH9Zkzl/5aBgM0aKDmQTaZSraGDdVt+f3lt7p1VRiXD2tfXxW6Hi57OUrUAlaDPC0tjdDQUACCg4PJyMio8Jy8vDyWLVvGhg0bbF/hFQoPh82b4fvv1cVP4SQ0TV3cKA7oS91WdoZcv74K4IAAtbVvr26L91V2W7++OuMUwglZDfKcnBxMJpPlsaenJ4WFhXh5lbx08+bNDBw4kICAAPtUeQVKd0OUIK+FLlyAgwchI6Nk+/lnFdI5OWWf6+EBzZqVzIwWHq7ut2pVctu0qfpTTAg3YjXITSYTuaXOesxmc5kQB/jwww9ZunSp7auzgebNIThYTWv77LN6V+PGCgrgp5/KBnZGhtpXfOHO2xs6dYKuXdVvYAlpIarFapCHhISQkpLCoEGDSE9PJygoqMzxc+fOkZ+fT7NmzexW5JUKD4d//Us1kTZooHc1buD33+Hbb8sG9sGDKsxBnVl36ADdukFkpLrt1k01gUhQC1FjVoM8LCyM1NRUIiMj0TSNmJgY4uLiCAwMpH///hw5coQWLVo4otbLduedMG+eaiufMEHvalzQX39BSopa9To5WTWNFLvmGhXSgwaVBHanTuoioBDCJgya5thppSIiIkhMTHTkW6Jp0KOHGgexf7/0XrliublqwvfkZBXe6enqQ/b3h1tugf79oXdv6NJF7RNCXLGqstOlBwQVMxhgyhR48EGVPWFhelfkZAoK4JtvSs64v/xS7TMa1ZzBc+aoVa979VJ9lIUQDuU2/+tGjlQXOxcvliCvlh9+gJ07VXB/+imcO6d+I/booX4r3n473HQT1Kmjd6VCuD23CXIfH3jsMZg1Cw4cgM6d9a6oFsrNhfh4WLlSnYGDugA5apRqLrn1VtXnWghRq7jVcLNHHlGB/soreldSy2RkwJNPqr6a48er/ttLlsCvv8KPP8KKFXD33RLiQtRSbhXkTZrA6NGwbp0ase3WLlyADRsgNFQNrlm1CoYMgd27ITMTJk+Gv+fUEULUbm4V5KDy6fx5lVtu6fBhePppNWH7mDHw559qrt+sLHj7bRXs0q1HCKfidkF+7bXqOt2rr6qZQt1Cfj68+65q5+7YUbUt3XabupB56JAK9kaN9K5SCHGZ3C7IAZ56Sk2S9+67eldiZ0eOwHPPqeHt996rBuq89JIaeVkc7DIrnxBOz216rZQ2cKA6MV28GO67zwVbEk6fhpkzVfuRwaDavidOhAEDZIY/IVyQW56OeXiortBpaZCaqnc1NqRpsHGjGgL/xhswaZLqefLBB2qIvIS4EC7JLYMc1IpLV12lzspdws8/qzPu++6DwEDVD3zJEtWsIoRwaW4b5HXqqNaG999XTclOKz8fYmLUZFRffQXLlqnbkBC9KxNCOIjbBjnAE0+oZpZaOpW6dZ99pobMP/+8agc/cED9o6QJRQi34tZB3qKF6szx5ptw9qze1dRAdraaj7dfPzWsfssW1Qullk8nLISwD7cOclBdEc+dgzVr9K6kGjQN1q9XFzPXroXp09UozMGD9a5MCKEjtw/yXr3UJH5Ll0JRkd7VVOHwYTWSaexYaNcO9u2D+fPV6u1CCLfm9kEO6qz8yBHVS6/WuXhRzffdvbvqL7liheoz2b273pUJIWoJqwOCzGYzs2fP5tChQxiNRqKjo2ldajKlTz/9lOXLl6NpGl27dmXWrFkYnGyEzbBhakWyxYshIkLvakr5+mt1Bn7okFrbcvFitQCxEEKUYvWMPDk5mfz8fOLj45k2bRqxsbGWYzk5OSxYsICVK1fy7rvv0qJFC06fPm3Xgu3B01ONnfn8c9i7V+9q/vbxx2o+lAsXYNs2NdBHQlwIUQmrQZ6WlkZoaCgAwcHBZGRkWI59++23BAUFMX/+fO677z4aNWpEQECA/aq1o/Hj1fKSS5boXQmQlKQuYLZpo/qEDxigd0VCiFrMapDn5ORgMpksjz09PSksLATg9OnT7Nmzh6effpo33niDt956iyNOOrqmXj0V5vHxakZX3SQmqraeLl3gk0/kLFwIYZXVIDeZTOTm5loem81mvP5eYLdBgwZce+21NG7cmLp169KrVy8OHDhgv2rtbNIkMJth+XKdCnjnHdWxvVcvtdCxrMgjhKgGq0EeEhLC7t27AUhPTycoKMhyrGvXrhw+fJjs7GwKCwv57rvvaN++vf2qtbM2beCf/4TXX4e8PAe/+erVavmifv1gxw5o0MDBBQghnJXVXithYWGkpqYSGRmJpmnExMQQFxdHYGAg/fv3Z9q0aUyYMAGAgQMHlgl6ZzRlimrdWLdOrfHpEEuXqqWLwsPhvffAz89BbyyEcAUGTdM0R75hREQEiYmJjnzLGtE0uP56NfI9M9MB6y7Exqq5w++6S/VM8fGx8xsKIZxRVdkpA4LKMRjUAKGDB2H7dju+kaZBVJQK8fvug4QECXEhxGWRIK/EPfdA8+Z2nKtc09Q6mXPnqsmv1q0DL7dcrEkIYQMS5JUwGtVssDt3Qqlu87ZhNsNjj8GiRfDkk+rKqkw7K4S4AhLkl/Dww+qao00HCBUWwrhxsHIlPPusWs1eFj8WQlwhSZFLaNhQTXOyYQOcOGGDL1hQAKNGwVtvqSaVmBgXXPVZCKEHCfIqTJmiJh98/fUr/EIXLsDw4eqC5sKF8H//JyEuhLAZCfIqdOqkunYvX64C/bLk5cHQofDhh/DaazB1qk1rFEIICXIrnnoKjh9XLSI1Zjar/uG7dqkVfR591NblCSGEBLk1t9+uVhCaPh2OHavhi5cvV8Ptly+H+++3S31CCCFBboXBAHFx6lrlhAmqC3i1/PgjzJgBgwbBxIl2rVEI4d4kyKuhfXtYsECN9Fy9uhovKCpSZ+C+vvDGG3JhUwhhVxLk1fTII9C/v7pWaXXK9UWL4Msv4dVX1RBRIYSwIwnyavLwgDVr1Mn1gw+q65iVysxU3QsjImDkSIfWKIRwTxLkNRAYqEZ6fvqpOtmuoKBANanUr69Wu5cmFSGEA0iQ19CDD6rlNJ99Fg4fLndw3jxIS1ND8Js00aU+IYT7kSCvIYNBXb/09VUn30VFfx/Yt08NvR81SjWrCCGEg1idO9VsNjN79mwOHTqE0WgkOjqa1q1bW45HR0ezb98+6tatC8Brr72Gv7+//SquBZo1U13D77sPXn4ZZky5qFK9cWNYtkzv8oQQbsZqkCcnJ5Ofn098fDzp6enExsayYsUKy/HMzExWr15NQECAXQutbSIj1ZJwUVEw7sfZNM7IgI8+gquu0rs0IYSbsdq0kpaWRmhoKADBwcFklJqg22w2c/ToUaKiooiMjGTz5s32q7SWMRjU1Cm31fmKgDf/hXncBDX4RwghHMzqGXlOTg4mk8ny2NPTk8LCQry8vMjLy2P06NE8+OCDFBUVMXbsWLp160anTp3sWnRt0bhuHgl17ufYmZa802QhM/UuSAjhlqyekZtMJnJzcy2PzWYzXn8vS+bn58fYsWPx8/PDZDLRu3dvDh48aL9qa5vnnsP/j8O8c3scUS/XIy1N74KEEO7IapCHhISwe/duANLT0wkKCrIc+/XXXxk5ciRFRUUUFBSwb98+unbtar9qa5NPPlEr/DzxBI8k3EaTJup652VPdyuEEJfJatNKWFgYqampREZGomkaMTExxMXFERgYSP/+/Rk2bBj33nsv3t7eDBs2jA4dOjiibn2dO6c6lLdvD7GxXFUX3nxTzV0+axbExupdoBDCnRg0rdrz+dlEREQEiYmJjnxL23vkEdWZ/LPP4B//sOx++GEV6J9/Dn366FifEMLlVJWdMiCoprZvV2u/TZtWJsRBreIWGKiaWPLydKpPCOF2JMhr4swZGD8eunSBOXMqHPb3VxNr/fgjzJQuLEIIB5Egr4nJk+HPP2HdOjVGvxK33gqTJsHSpZCS4uD6hBBuSYK8uj74QAX4889Dz55VPnXePOjQQV0PPXfOQfUJIdyWBHl1/PWXupIZHKyC3Io6ddRay7//rprShRDCniTIq+Pxx+H0aXVGbjRW6yX/+Ac884zq3LJ1q53rE0K4NQlya9LSICEBXngBrr22Ri998UX1khEj1GIUQghhDxLk1ixerLqjTJ5c45f6+MC2bdCqlRostH27HeoTQrg9CfKqZGVBfLzqcliv3mV9iebN1dl4x44wdKi6ZiqEELYkQV6V5cvVEkBPPnlFX6ZxY/j4Y+jRA4YPV78bhBDCViTILyUvT43g/Oc/oW3bK/5yV10FO3dC375qZaG1a6/4SwohBCBBfmnr10N2Njz1lM2+pL8/JCXB7berPuavvWazLy2EcGMS5JUxm2HJEjXw56abbPql69SB//xHtZc//rha81MIIa6E1Wls3dL27XDwIGzYoNZ0szEfH9i8GcaMUX3N8/JU70Y7vJUQwg1IkFdm8WJo1gzuucdub+HtDW+/DX5+ag7z3Fw1j7mEuRCipiTIy8vIUFclX3qp2qM4L5enp5q/vE4d+Ne/1Jn5K6+AhzR4CSFqQIK8vFdeUafJEyc65O08PODVV1WYv/yyOjN/4w0V8kIIUR1Wz/3MZjNRUVGMGDGCMWPGcPTo0UqfM2HCBDZu3GiXIh3m5EnVW2XsWGjY0GFvazCoM/JZsyAuDkaPhoICh729EMLJWT0jT05OJj8/n/j4eNLT04mNjWXFihVlnrNkyRLOnj1rtyIdZuVKtXryZQzHv1IGA8yerc7MZ8yA8+fVwCEfH4eXIoRwMlbPyNPS0ggNDQUgODiYjIyMMse3bduGwWCwPMdpXbyoRnIOHAidO+tWxvTpsGyZGso/bJgsGSeEsM5qkOfk5GAymSyPPT09KSwsBODw4cNs2bKFyTqcwdrcpk1w/LhNBwBdrieeUBdBd+xQg4cOH9a7IiFEbWa1acVkMpGbm2t5bDab8fJSL3v//fc5fvw4999/P1lZWXh7e9OiRQv69etnv4rtQdNUl8OuXSEsTO9qABg3To0EfeghNRXuc8/Bs89KU4sQoiKrQR4SEkJKSgqDBg0iPT2doKAgy7Hp06db7i9btoxGjRo5X4iDmp7wu+9Ud5Fa1JH7nnsgNBSmTlXt5++8o5rxb71V78qEELWJ1aaVsLAwjEYjkZGRzJs3j5kzZxIXF8euXbscUZ9jLF4MjRrBqFF6V1JB06YqwLdvh8JCuO02eOABtfqcEEIAGDRN0xz5hhERESQmJjryLav2008QFAT/938wZ47e1VTp/HmIjlZdFevVU/3OH3igVv0RIYSwk6qyU8YQvvIKeHnBo4/qXYlVfn5qwGl6OnTpotrRb7kFDhzQuzIhhJ7cO8jPnFEjcEaOVHOrOImuXVWz/urVsH8/XHedmnTr/Hm9KxNC6MG9g/yNN9SY+FrQ5bCmPDzUCnQHD0JkpGpy6d4dkpP1rkwI4WjuG+SFhWrkzS23QHCw3tVctiZNYN26kgAPC1PXbI8f17cuIYTjuG+QJybC77875dl4Zfr3V80sUVHw7rvQqVPJjANCCNfmvkG+eDG0aweDB+tdic34+sKLL8L336t280cfhZYt1bD/n37SuzohhL24Z5B/9ZXaJk92yfliO3WClBTYtk0NKFq0CDp0gDvuUH+IyMyKQrgW9wzyxYuhfn21ArKLMhhgwAAV3L/9prrIHzgAw4dDYKDq5fLbb3pXKYSwBfcL8t9+g/feU5OYlJoMzJU1b66C+8gRtfBzz56qP3qbNjBkCGzZAkVFelcphLhc7hfkr76qbp98Ut86dODlBXfeqYL7yBGYORPS0tS+Nm1g7lz44w+9qxRC1JR7BXlODqxaBRERqn3BjbVurfqe//YbbN4MHTuqHi+Bgar5ZccOaUsXwlm4V5CvXQv/+5/LdDm0BW9vFdw7d8KPP6qZFj/9VLWvN2yoztZfeUWtSe3YWXmEENXlPkFuNqtEuvFG6NNH72pqpfbt1YRcWVnqMsKoUWrk6JQpak705s3Vvrg4uVAqRG1idT5yl7Fli+pMHR2tdyW1no+Pan2KiFCPjx6FXbtKtnfeUfvbt1crGPXvr+ZId+B61UKIUtwnyJcsgVatVDuCqJHWrdVMi+PGqeaVzEwV6MnJ8PbbagSpwQA9eqhQv/12uOEGaNBA78qFcA/uEeSZmWqEzPz5quuGuGwGA3TrprbJk9UF0b17Vajv2qV+Xy5YoJ7bsmXJc7t2VbedO0Pdurr+E4RwOe6Rah98oG7HjNG3Dhfk7a0uOfTpo/qq5+ZCaqqaMz0jQ20pKSVzvhgM0LZt2XDv1k31mjEadf2nCOG0rAa52Wxm9uzZHDp0CKPRSHR0NK1bt7Ycf/vtt0lMTMRgMDBu3DgGDRpk14IvS1IShIQ41ZzjzqpuXTUVwB13lOwrKoKff1ahnplZEvAffaQmoQT1h1KHDirUg4JUK1jLliW3V10lKyEJcSlWgzw5OZn8/Hzi4+NJT08nNjaWFStWAJCdnc3GjRv597//zcWLFxk8eDDh4eEYatP/uNOn4Ysv1OgXoQtPTxXOQUElF1AB8vPh8OGSYM/IgH371LQC5Uea1qlTNtiLbyXshahGkKelpREaGgpAcHAwGRkZlmMBAQG8//77eHl5kZWVhY+PT+0KcVAdpM1mCA/XuxJRjtFY0rRSWmGhmk/999/VduxY2dtdu9QIVLO57Ovq1FFdJBs2hICA6t3WqyfhL5yf1SDPycnBVGpOEk9PTwoLC/H6+6Khl5cXGzZsYNmyZYypjW3QSUnqVO3GG/WuRFSTlxe0aKG23r0rf05hIfz5Z8Wg/+MPyM5WvwgOHIBTp+DcuUu/l6enCvXizd9fTcFTfrvU/tKbn5/65SS/GISjWQ1yk8lEbm6u5bHZbLaEeLHRo0dz77338tBDD/HVV1/R+1L/+xzNbFZBfscd0lvFxXh5lTStWFNQoMI9O1sF+6VuT59WA3+zstRsDsVbTRfn8PFRc8MXb+UfV7bfaFQXjou38o/Lb6WPe3mpzdOzZCv9uKpjxZuHR+Vb6WPyC6r2sppuISEhpKSkMGjQINLT0wkKCrIc++WXX1i0aBHLli3D29sbo9GIh0ctGiyanq5OzaRZxa15e8PVV6vtchQUqN44pcM9J0ed6Zd+fOFC2e3ixcr3nT0LJ06U3Z+fr96neKuts1FWFvYGg9qK71/q9lL7in9BlH5sbSv9/OL7Nd1X7EruW3tc+r6fn1pdskOHip/rlbIa5GFhYaSmphIZGYmmacTExBAXF0dgYCD9+/enU6dOjBgxAoPBQGhoKDfccIPtq7xcSUnqduBAfesQTs3bWw1ucuQAJ7NZNR8VB3v5oC+9FRaq4C9/W937ZnPF7VL7yz9H09R9TSt7/1K3xfdLP4aS+9Xdil9T/ra6+4pdyX1rj8sfs+dfNAZNc+xUSBERESQmJjrmzfr2VadAe/c65v2EEMJOqsrOWtQOYmPZ2Wo5N2lWEUK4ONcN8uJuh7VxgJIQQtiQ6wb51q2qP1ltarMXQgg7cM0gN5vVEvJ33KH6TwkhhAtzzSD/9lvVv0uaVYQQbsA1g3zrVnU7YIC+dQghhAO4ZpAnJUGvXtCkid6VCCGE3blekJ86BXv2SLOKEMJtuF6Q79ghsx0KIdyK6wV5UpKan/T66/WuRAghHMK1gry42+GAAdLtUAjhNlwryNPS4ORJaVYRQrgV1wrypCQ1xZh0OxRCuBHXC/Lrr4fGjfWuRAghHMZ1gvyvv1S3Q2lWEUK4GdcJ8h071EzuEuRCCDfjOkGelASNGqkRnUII4UasLvVmNpuZPXs2hw4dwmg0Eh0dTevWrS3H165dy0cffQTAzTffzBNPPGG/ai9dpHQ7FEK4Latn5MnJyeTn5xMfH8+0adOIjY21HPv999/5z3/+w6ZNm0hISODzzz/n4MGDdi24Unv3qjZyaVYRQrghq2fkaWlphIaGAhAcHExGRoblWNOmTVm9ejWef58FFxYW4uPjY6dSqyDdDoUQbszqGXlOTg4mk8ny2NPTk8LCQgC8vb0JCAhA0zTmz59Ply5daNOmjf2qvZStW9VKQI0aOf69hRBCZ1aD3GQykZuba3lsNpvx8io5kb948SJPP/00ubm5zJo1yz5VVuXkSfjmG2lWEUK4LatBHhISwu7duwFIT08nKCjIckzTNB577DE6duzInDlzLE0sDlXc7VCmrRVCuCmrbeRhYWGkpqYSGRmJpmnExMQQFxdHYGAgZrOZr7/+mvz8fD777DMApk6dSo8ePexeuMXWrWokZ8+ejntPIYSoRawGuYeHB3PmzCmzr127dpb7+/fvt31V1VVUBNu3q2YVD9fpEi+EEDXh3Om3d69aEUiaVYQQbsy5g3zrVnUmfscdelcihBC6ce4gT0pS3Q4bNtS7EiGE0I3zBvmJE6ppRZpVhBBuznmDfPt2me1QCCFw5iBPSoImTSAkRO9KhBBCV84Z5MXdDgcOlG6HQgi355wp+PXXkJ0tzSpCCIGzBnlSknQ7FEKIvzlvkPfuDQEBelcihBC6c74gP35cdTuUZhUhhACcMci3b1e3EuRCCAE4Y5AnJcHVV4MjZ1gUQohazLmCvLBQuh0KIUQ5zpWGX38Np09Ls4oQQpTiXEEu3Q6FEKICq0FuNpuJiopixIgRjBkzhqNHj1Z4TnZ2NgMGDODixYt2KdJi61bo0weuusq+7yOEEE7EapAnJyeTn59PfHw806ZNIzY2tszxzz77jHHjxnHy5Em7FQnAn3/Cvn3SrCKEEOVYDfK0tDRCQ0MBCA4OJiMjo+wX8PAgLi6OBg0a2KVAi5QUdSvT1gohRBlW1+zMycnBZDJZHnt6elJYWIiXl3pp37597Vddab16QXQ0BAc75v2EEMJJWA1yk8lEbm6u5bHZbLaEuEN16ADPP+/49xVCiFrOatNKSEgIu3fvBiA9PZ2goCC7FyWEEKL6rJ5ah4WFkZqaSmRkJJqmERMTQ1xcHIGBgfTv398RNQohhKiC1SD38PBgzpw5Zfa1a9euwvM+/vhj21UlhBCi2pxrQJAQQogKJMiFEMLJSZALIYSTkyAXQggnJ0EuhBBOzuEje7KysoiIiHD02wohhFPLysq65DGDpmmaA2sRQghhY9K0IoQQTk6CXAghnJwEuRBCODkJciGEcHIS5EII4eQkyIUQwsnpsEKEdWazmdmzZ3Po0CGMRiPR0dG0bt3acjwhIYFNmzbh5eXFo48+yq233uqQugoKCnjuuefIysoiPz+fRx99tMxUvmvXruXdd98lICAAgBdffJG2bds6pLa77rrLspJTy5YtmTdvnuWYXp9XYmIi//73vwG4ePEiBw4cIDU1lXr16gEQHR3Nvn37qFu3LgCvvfYa/v7+dq3pu+++4+WXX2b9+vUcPXqUZ599FoPBQIcOHZg1axYeHiXnNhcuXOCZZ57h1KlT1K1bl/nz51u+t/as68CBA8ydOxdPT0+MRiPz58+nUaNGZZ5f1ffbXnX98MMPTJw4kWuuuQaAkSNHMqjU0ot6fV5PPfUUf/31F6D6Wl933XUsXrzY8lxN0+jXr5+l7uDgYKZNm2bTeirLhvbt2zvu50urhbZv367NmDFD0zRN+/bbb7VHHnnEcuzEiRPakCFDtIsXL2pnz5613HeEzZs3a9HR0Zqmadrp06e1m2++uczxadOmafv373dILaVduHBBGzZsWKXH9Py8Sps9e7a2adOmMvsiIyO1U6dOOayGVatWaUOGDNHuueceTdM0beLEidpXX32laZqmvfDCC9qOHTvKPH/NmjXa0qVLNU3TtC1btmhz5851SF2jRo3SfvjhB03TNG3jxo1aTExMmedX9f22Z10JCQnam2++ecnn6/V5FTtz5ow2dOhQ7fjx42X2//rrr9rEiRPtUkuxyrLBkT9ftbJppaoFn7///nt69OiB0WjE39+fwMBADh486JC6Bg4cyOTJkwH1W97T07PM8czMTFatWsXIkSN5/fXXHVITwMGDBzl//jzjxo1j7NixpKenW47p+XkV279/Pz/99BMjRoyw7DObzRw9epSoqCgiIyPZvHmz3esIDAxk2bJllseZmZnccMMNAPTr148vvviizPNL/xz269ePL7/80iF1LVq0iM6dOwNQVFSEj49PmedX9f22Z10ZGRl88sknjBo1iueee46cnJwyz9fr8yq2bNkyRo8eTZMmTcrsz8zM5Pjx44wZM4aHHnqIX375xeY1VZYNjvz5qpVBfqkFn4uPlf7zu27duhV+oOylbt26mEwmcnJymDRpElOmTClzfPDgwcyePZu33nqLtLQ0UlJSHFKXr68v48eP58033+TFF1/k6aefrhWfV7HXX3+dxx9/vMy+vLw8Ro8ezYIFC1i9ejXvvPOO3X/BDBgwoMx6s5qmYTAYAPW5nDt3rszzS392lR23V13FQbRv3z42bNjAAw88UOb5VX2/7VlX9+7dmT59Om+//TatWrVi+fLlZZ6v1+cFcOrUKb788stKp/9o3LgxDz/8MOvXr2fixIk888wzNq+psmxw5M9XrQzyqhZ8Ln8sNzfX7u2qpf33v/9l7NixDBs2jDvvvNOyX9M07r//fgICAjAajdx888388MMPDqmpTZs2DB06FIPBQJs2bWjQoAEnT54E9P+8zp49y5EjR+jdu3eZ/X5+fowdOxY/Pz9MJhO9e/d2+F8Kpdsrc3NzLW33xUp/dpUdt6etW7cya9YsVq1aVaHdtKrvtz2FhYXRrVs3y/3yP996fl7btm1jyJAhFf5KBujWrZvlWlavXr04ceIEmh1mJimfDY78+aqVQV7Vgs/du3cnLS2Nixcvcu7cOX7++WeHLQj9119/MW7cOJ555hnuvvvuMsdycnIYMmQIubm5aJrGnj17LD/09rZ582ZiY2MBOH78ODk5OTRu3BjQ9/MC+Oabb+jTp0+F/b/++isjR46kqKiIgoIC9u3bR9euXR1WF0CXLl3Ys2cPALt376ZXr15ljoeEhPDpp59ajvfs2dMhdX3wwQds2LCB9evX06pVqwrHq/p+29P48eP5/vvvAfjyyy8rfL/0+ryK6+nXr1+lx1599VXeeustQDVLNWvWzHKmbCuVZYMjf75q5aRZxb1WDh8+bFnweffu3ZYFnxMSEoiPj0fTNCZOnMiAAQMcUld0dDRJSUlleqLcc889nD9/nhEjRvD++++zfv16jEYjffr0YdKkSQ6pKz8/n5kzZ/LHH39gMBh4+umn+e6773T/vABWr16Nl5eXpXmg9MLdq1evJikpCW9vb4YNG8bIkSPtXs+xY8eYOnUqCQkJHDlyhBdeeIGCggLatm1LdHQ0np6ejBs3jpUrV1JUVMSMGTM4efIk3t7eLFy40G6BWVzXxo0b6dOnD82aNbOcoV1//fVMmjSJ6dOnM2XKFBo1alTh+x0SEmLXuhISEsjMzGTu3Ll4e3vTqFEj5s6di8lk0vXzSkhIAFSz5saNG8uc1RbXdf78eZ555hny8vLw9PQkKiqq0nWHr0Rl2fD8888THR3tkJ+vWhnkQgghqq9WNq0IIYSoPglyIYRwchLkQgjh5CTIhRDCyUmQCyGEk5MgF0IIJydBLoQQTu7/AcKUqMcA4eToAAAAAElFTkSuQmCC","text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{},"output_type":"display_data"},{"data":{"text/plain":["array([0.28582562, 0.71417438])"]},"execution_count":27,"metadata":{},"output_type":"execute_result"}],"source":"get_matrix_pow([0.9, 0.1], A, n=20)"},{"cell_type":"code","execution_count":17,"metadata":{"collapsed":false,"id":"D8D420086F914A7AA74AC2CD9BCFA674","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"data":{"text/html":["<img src=\"https://cdn.kesci.com/upload/rt/D8D420086F914A7AA74AC2CD9BCFA674/rjvlmhpqbq.png\">"],"text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{},"output_type":"display_data"},{"data":{"text/plain":["array([0.28568063, 0.71431937])"]},"execution_count":17,"metadata":{},"output_type":"execute_result"}],"source":"get_matrix_pow([0.1,0.9],A,n = 20)"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"7572341CBFA144238A5888ACE1B05223","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"我们可以观察到无论初始状态如何，只要状态转移一直持续下去，处在每个状态的概率会服从一个平稳分布。\n"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"0BE10B52F6D643BF89AAE66620E3CFDD","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"假定我们可以得到我们需要采样样本的平稳分布所对应的马尔科夫链状态转移矩阵，那么我们就可以用马尔科夫链采样得到我们需要的样本集，进而进行蒙特卡罗模拟。\n\n但是我们该如何得到马尔可夫链的状态转移矩阵呢？这就需要MCMC。"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"0104DD102DE94C0EA60835BC596164DF","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"#### MCMC"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"57B6F8167F354C2B818229E78583560F","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"MCMC，即马尔可夫蒙特卡洛方法，是一组利用马尔可夫链从随机分布中取样的算法。生成的马尔科夫链就是对于目标分布的近似估计。"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"1F88D894CF8F462A8ABDD0DED1DEE7A5","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"##### MCMC采样算法如下：\n\n1. 输入任意给定的马尔科夫链状态转移矩阵Q，目标平稳分布$\\pi(x)$  ，设定状态转移次数阈值 $n_1$ ，需要的样本数 $n_2$ ;\n\n2. 从任意简单概率分布得到初始状态值 $x_0$ ；\n\n3. for t = 0 in n1 + n2 - 1 \n\n- 从条件概率分布$Q(x|x_t)$得到样本值$x_*$\n\n- 从均匀分布中采样 $U \\sim [0,1]$\n\n- 如果 $u<\\alpha(x_t,x_*)=\\pi(x_*)Q(x_*,x_t)$,则接受$x_t -> x_*$, 即$x_{t+1}=x_*$ \n\n- 否则不接受转移，$t =max{t-1,0}$"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"202F488A7B8E4B25B96C56499D14D797","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"但是这个采样算法还是比较难在实际中应用，因为在第三步中，由于 $\\alpha(x_t,x_*)$ 可能非常的小，比如0.1，导致我们大部分的采样值都被拒绝转移，采样效率很低。\n\n有可能我们采样了上百万次马尔可夫链还没有收敛，也就是上面这个 $n_1$ 要非常非常的大，这让人难以接受，怎么办呢？这时就轮到我们的M-H采样出场了。"},{"cell_type":"code","execution_count":18,"metadata":{"collapsed":false,"id":"8B4C8EF2408546AB8BFE1C536CA41745","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[],"source":"def norm_dist_prob(theta,loc=3,scale=2): #定义目标分布\n    y = norm.pdf(theta, loc=3, scale=2)\n    return y\n\nT = 5000 #采样次数5000\npi = [0 for i in range(T)] #用来存储采样点的数组\nsigma = 1 #标准差\nt = 0 #当前采样次数\nwhile t < T-1:\n    t = t + 1\n    pi_star = norm.rvs(loc=pi[t - 1], scale=sigma, size=1, random_state=None)   #状态转移进行随机抽样\n    alpha = (norm_dist_prob(pi_star[0])*norm_dist_prob(pi_star[0],loc = pi[t - 1]))   #alpha值\n\n    u = np.random.uniform(0, 1)\n    if u < alpha:\n        pi[t] = pi_star[0]\n    else:\n        pi[t] = pi[t - 1]"},{"cell_type":"code","execution_count":19,"metadata":{"collapsed":false,"id":"58108FB8DEEF4EF682AB458172D60435","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"data":{"text/html":["<img src=\"https://cdn.kesci.com/upload/rt/58108FB8DEEF4EF682AB458172D60435/rjvm8kd43g.png\">"],"text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":"plt.plot(range(5000),pi)\nplt.show()"},{"cell_type":"code","execution_count":20,"metadata":{"collapsed":false,"id":"0A1AC6E139D54545860211A1724DAA51","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"data":{"text/html":["<img src=\"https://cdn.kesci.com/upload/rt/0A1AC6E139D54545860211A1724DAA51/rjvm8xpoze.png\">"],"text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":"plt.scatter(pi, norm.pdf(pi, loc=3, scale=2),label='Target Distribution')\nnum_bins = 50\nplt.hist(pi, num_bins, density = True, facecolor='red', alpha=0.7,label='Samples Distribution')\nplt.legend()\nplt.show()"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"5DFE2CC22D2042F29F949CB0AA039633","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"##### M-H采样算法\n\nM-H采样是Metropolis-Hastings采样的简称，这个算法首先由Metropolis提出，被Hastings改进，因此被称之为Metropolis-Hastings采样或M-H采样。"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"A428A4F8FAA84E9F8D0F29B322962032","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"M-H采样解决了我们上一节MCMC采样接受率过低的问题。\n\n我们可以取$\\alpha(i,j)=min(\\frac{\\pi(j)Q(j,i)}{\\pi(i)Q(i,j)},1 )$ "},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"CB179A17562D43FD900B92BB6942A754","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"M-H 采样算法如下：\n\n1. 输入任意给定的马尔科夫链状态转移矩阵Q，目标平稳分布$\\pi(x)$  ，设定状态转移次数阈值 $n_1$ ，需要的样本数 $n_2$ ;\n\n2. 从任意简单概率分布得到初始状态值 $x_0$ ；\n\n3. for t = 0 in n1 + n2 - 1 \n\n- 从条件概率分布$Q(x|x_t)$得到样本值$x_*$\n\n- 从均匀分布中采样 $U \\sim [0,1]$\n\n- 如果 $u<\\alpha(x_t,x_*)=min(\\frac{\\pi(j)Q(j,i)}{\\pi(i)Q(i,j)},1 )$,则接受$x_t -> x_*$, 即$x_{t+1}=x_*$ \n\n- 否则不接受转移，t=max{t-1,0}\n"},{"cell_type":"code","execution_count":21,"metadata":{"collapsed":false,"id":"55AC3ABDD3B1412899F81583F6B585C8","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[],"source":"def norm_dist_prob(theta):#定义目标分布\n    y = norm.pdf(theta, loc=3, scale=2)\n    return y\n\nT = 5000 #采样次数5000\npi = [0 for i in range(T)] #用来存储采样点的数组\nsigma = 1 #标准差\nt = 0 #当前采样次数\nwhile t < T-1:\n    t = t + 1\n    pi_star = norm.rvs(loc=pi[t - 1], scale=sigma, size=1, random_state=None)   #状态转移进行随机抽样\n    alpha = min(1, (norm_dist_prob(pi_star[0]) / norm_dist_prob(pi[t - 1])))   #alpha值\n\n    u = np.random.uniform(0, 1)\n    if u < alpha:\n        pi[t] = pi_star[0]\n    else:\n        pi[t] = pi[t - 1]\n\n"},{"cell_type":"code","execution_count":22,"metadata":{"collapsed":false,"id":"D7A4B9C8106044C68ABD158C33938C60","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"data":{"text/html":["<img src=\"https://cdn.kesci.com/upload/rt/D7A4B9C8106044C68ABD158C33938C60/rjvmafnf13.png\">"],"text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":"plt.plot(range(5000),pi)\nplt.show()"},{"cell_type":"code","execution_count":23,"metadata":{"collapsed":false,"id":"6BA870C2EE6A4698ACD5C3FDA6CFC995","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"data":{"text/html":["<img src=\"https://cdn.kesci.com/upload/rt/6BA870C2EE6A4698ACD5C3FDA6CFC995/rjvmasrq7s.png\">"],"text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":"plt.scatter(pi, norm.pdf(pi, loc=3, scale=2),label='Target Distribution')\nnum_bins = 50\nplt.hist(pi, num_bins, density = True, facecolor='red', alpha=0.7,label='Samples Distribution')\nplt.legend()\nplt.show()"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"AF90D6038AB04E7AB763AB1B748CA7D3","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"## Part 2: MCMC原理"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"0825E217238B45419A914D24078342FC","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"使用MCMC的原因：\n- 由于贝叶斯公式分母难以计算\n- 由于参数过多后验分布形态复杂\n\n在本节课中，我们首先回顾如何计算后验，以及计算后验面临的问题；\n\n之后，我们介绍如何通过采样的方法解决现有的问题。"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"C603A4EC7EA740268D657964C05A05E5","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"### 计算后验面临的问题"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"6C58CA44B5B642AD884DEB4DE6779996","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"**以探究地球海水覆盖率为例。**\n\n- 假设我们获得数据：抛地球仪10次，其中7次海面朝上。\n- 在获得数据的同时，我们以二项分布函数作为**似然函数**。\n- 假设我们在获得数据以前的**先验**认为：地球海水覆盖率为0.5。\n\n我们的任务是根据数据更新我们的先验。"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"D5BCE1888E024400BE14AB41B1C8BFAB","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"1. 假设我们获得数据：抛地球仪10次，其中7次海面朝上。"},{"cell_type":"code","execution_count":25,"metadata":{"collapsed":false,"id":"05E1D9BA94DA4C468CD72FF34CAA9225","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[],"source":"import numpy as np               # numpy 是专门用于数组运算的包\nimport scipy.stats as st         # 从scipy.stats里载入分布函数\nimport matplotlib.pyplot as plt  # matplotlib.pyplot 是专门用于画图的包\nimport seaborn as sns            # seaborn是专门用于绘制统计数据的包\nimport arviz as az               # arviz 是专门用于探索贝叶斯模型的包"},{"cell_type":"code","execution_count":26,"metadata":{"collapsed":false,"id":"E53A7E4539D14F918C3DA493A0BAC46B","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["数据： [1, 1, 1, 1, 1, 0, 0, 1, 1, 0]\n"]}],"source":"data = [1,1,1,1,1,0,0,1,1,0] # 1代表海面朝上，0代表陆地朝上\nprint(\"数据：\",data)"},{"cell_type":"code","execution_count":27,"metadata":{"collapsed":false,"id":"3EA9C49E7F81428B968602B0C8D782B4","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"data":{"text/plain":["(-1.0, 2.0)"]},"execution_count":27,"metadata":{},"output_type":"execute_result"},{"data":{"text/html":["<img src=\"https://cdn.kesci.com/upload/rt/3EA9C49E7F81428B968602B0C8D782B4/rjvmd9ti5u.png\">"],"text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":"plt.hist(data)\nplt.xticks([0,1],labels=[\"earth\",\"water\"])\nplt.xlim(-1,2)"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"9623E344118A440484B3E38C0AEDA134","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"2. 我们的先验：地球海水覆盖率为0.5。\n  \n由于我们的先验不是分布，而是一个值，因此可以认为我们对先验的信心为100%，即 $prior=1$"},{"cell_type":"code","execution_count":29,"metadata":{"collapsed":false,"id":"D3D7F17D811A46D5A515557B0D450E27","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["先验：地球海水覆盖率为： 0.5\n"]}],"source":"prior_θ = 0.5\nprior=1 # 由于先验只有一个值，我们用1(100%)表示先验出现的概率\nprint(\"先验：地球海水覆盖率为：\",prior_θ)"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"9F74DAB011954511BF9CA7A4FA8F907B","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"3. 结合先验参数(prior_θ)与数据(data)计算似然值。\n\n我们选择伯努利分布函数(bernoulli)作为似然函数。"},{"cell_type":"code","execution_count":30,"metadata":{"collapsed":false,"id":"B56625EC5B2E47AFB2073F2F9382FD56","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["根据当前数据与先验得到的似然值： 0.0009765625\n"]}],"source":"likelihood  = st.bernoulli(prior_θ).pmf(data).prod()\nprint(\"根据当前数据与先验得到的似然值：\",likelihood)"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"4AD3F4A5FE7241828354C79DFA48B306","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"4. 计算后验，即通过似然值更新先验。\n\n注意：\n\n由于这里是没有考虑贝叶斯公式的分母部分，因此我们得到的是未标准化的 invalid 后验"},{"cell_type":"code","execution_count":31,"metadata":{"collapsed":false,"id":"DD1393CB57CD413BAD682DE8EF98244E","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["未标准化的后验： 0.0009765625\n"]}],"source":"unstd_posterior = likelihood * prior\nprint(\"未标准化的后验：\", unstd_posterior)"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"489D781F98464D0B8C06F67F25429C29","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"**练习：**\n\n当我们假设的先验不同时，似然与后验也会产生相应的变化。\n\n比如，我们尝试将先验(地球海水覆盖率)设为0.5，0.6，和0.7。"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"F4DC8721601B4B3592A83381DD91D6DB","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"- 为了后面方便计算，我们把计算未标准化后验的过程用一个函数包裹起来\n\n该函数的每一步计算过程与之前完全相同"},{"cell_type":"code","execution_count":32,"metadata":{"collapsed":false,"id":"8439CEB834E84C5882DA972C513889B4","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[],"source":"def calculate_posterior(prior_θ, data):\n    prior=1\n    likelihood  = st.bernoulli(prior_θ).pmf(data).prod()\n    unstd_posterior = likelihood * prior\n    \n    return unstd_posterior"},{"cell_type":"code","execution_count":33,"metadata":{"collapsed":false,"id":"4F1BD69B03EE43E38C64CBDFA95BD139","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["未标准化的后验： 0.0009765625\n"]}],"source":"prior_θ = 0.5 # 假设先验为0.5\nunstd_posterior = calculate_posterior(prior_θ,data)\nprint(\"未标准化的后验：\",unstd_posterior)"},{"cell_type":"code","execution_count":34,"metadata":{"collapsed":false,"id":"61D5602A69204000870A9CCC9431BB0E","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"ename":"TypeError","evalue":"'>=' not supported between instances of 'ellipsis' and 'int'","output_type":"error","traceback":["\u001b[0;31m---------------------------------------------------------------------------\u001b[0m","\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)","\u001b[0;32m/tmp/ipykernel_48/69097260.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     15\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     16\u001b[0m \u001b[0mprior_θ\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m...\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 17\u001b[0;31m \u001b[0munstd_posterior\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcalculate_posterior\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprior_θ\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mdata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     18\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"未标准化的后验：\"\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0munstd_posterior\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/tmp/ipykernel_48/69097260.py\u001b[0m in \u001b[0;36mcalculate_posterior\u001b[0;34m(prior_θ, data)\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mcalculate_posterior\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprior_θ\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      6\u001b[0m     \u001b[0mprior\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 7\u001b[0;31m     \u001b[0mlikelihood\u001b[0m  \u001b[0;34m=\u001b[0m \u001b[0mst\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbernoulli\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprior_θ\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpmf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprod\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      8\u001b[0m     \u001b[0munstd_posterior\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlikelihood\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0mprior\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      9\u001b[0m     \u001b[0;32mreturn\u001b[0m \u001b[0munstd_posterior\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/site-packages/scipy/stats/_distn_infrastructure.py\u001b[0m in \u001b[0;36mpmf\u001b[0;34m(self, k)\u001b[0m\n\u001b[1;32m    503\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    504\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mpmf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mk\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 505\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdist\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpmf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mk\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    506\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    507\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mlogpmf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mk\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/site-packages/scipy/stats/_distn_infrastructure.py\u001b[0m in \u001b[0;36mpmf\u001b[0;34m(self, k, *args, **kwds)\u001b[0m\n\u001b[1;32m   3141\u001b[0m         \u001b[0m_a\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0m_b\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_get_support\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3142\u001b[0m         \u001b[0mk\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0masarray\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mk\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0mloc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 3143\u001b[0;31m         \u001b[0mcond0\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_argcheck\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   3144\u001b[0m         \u001b[0mcond1\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mk\u001b[0m \u001b[0;34m>=\u001b[0m \u001b[0m_a\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m&\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mk\u001b[0m \u001b[0;34m<=\u001b[0m \u001b[0m_b\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m&\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_nonzero\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mk\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3145\u001b[0m         \u001b[0mcond\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcond0\u001b[0m \u001b[0;34m&\u001b[0m \u001b[0mcond1\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.7/site-packages/scipy/stats/_discrete_distns.py\u001b[0m in \u001b[0;36m_argcheck\u001b[0;34m(self, p)\u001b[0m\n\u001b[1;32m    129\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    130\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_argcheck\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mp\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 131\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mp\u001b[0m \u001b[0;34m>=\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m&\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mp\u001b[0m \u001b[0;34m<=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    132\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    133\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_get_support\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mp\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;31mTypeError\u001b[0m: '>=' not supported between instances of 'ellipsis' and 'int'"]}],"source":"import numpy as np               # numpy 是专门用于数组运算的包\nimport scipy.stats as st         # 从scipy.stats里载入分布函数\n\n# 定义计算后验的函数\ndef calculate_posterior(prior_θ, data):\n    prior=1\n    likelihood  = st.bernoulli(prior_θ).pmf(data).prod()\n    unstd_posterior = likelihood * prior\n    return unstd_posterior\n  \n########################################################\n# 练习\n# 请尝试修改先验参数 prior_θ 修改为0.6或者0.7\n########################################################\n\nprior_θ = ...\nunstd_posterior = calculate_posterior(prior_θ,data)\nprint(\"未标准化的后验：\",unstd_posterior)"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"72DA770A35EE4497B085830CDD5C8797","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"通过定义函数`calculate_posterior`，我们只需要输入先验参数和数据，就可以计算出后验后验(invalid)。\n\n经过尝试可以发现，当我们的先验参数越接近数据中海面出现的频率(0.7)时，后验值最大。\n\n但此时的后验值还很小，还不是一个有效的概率值，接下来我们结合 **有效的先验概率分布**和**贝叶斯公式分母(边际似然)**计算有效的后验概率分布。"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"BC48554CEDB84417AD8B1B5C67CD85FB","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"**计算valid 后验概率分布**\n\n如果我们假设，海水覆盖面率可能不止是一个值，而可能为0.6，0.7，0.8，其中0.7的可能性最大时：\n\n比如，我们认为海水覆盖率为0.7时的概率为0.5，而海水覆盖率为0.6和0.7的概率为0.25。\n\n此时先验概率的和为1，所以该分布是一个有效的(valid)概率分布。"},{"cell_type":"code","execution_count":35,"metadata":{"collapsed":false,"id":"C4EDA33B09DF426F8051EA5A4D1F23DF","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[],"source":"prior_θs = [0.6,0.7,0.8] # 假设先验为0.6，0.7，0.8\npriors = [0.25,0.5,0.25] # 假设我们对不同先验的信心为0.25,0.5,0.25"},{"cell_type":"code","execution_count":36,"metadata":{"collapsed":false,"id":"C296EEB6D4724318922A6E822E04466C","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"data":{"text/plain":["([<matplotlib.axis.XTick at 0x7fbfe2b95fd0>,\n","  <matplotlib.axis.XTick at 0x7fbfe2b95750>,\n","  <matplotlib.axis.XTick at 0x7fbfe2b7fb90>],\n"," [Text(0, 0, ''), Text(0, 0, ''), Text(0, 0, '')])"]},"execution_count":36,"metadata":{},"output_type":"execute_result"},{"data":{"text/html":["<img src=\"https://cdn.kesci.com/upload/rt/C296EEB6D4724318922A6E822E04466C/rjvmiow3kl.png\">"],"text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":"# 绘制先验分布的概率分布\nplt.plot(prior_θs,priors)\nplt.xlabel(\"Prior belief of the proportion of sea surface coverage\")\nplt.ylabel(\"Probability\")\nplt.xticks(prior_θs)"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"C936BF3955384AA682F8A6CB9C72DA09","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"接下来我们计算后验概率：\n- `calculate_posterior` 是我们定义的计算未标准化的后验\n- 因为先验有多个值(0.6,0.7,0.8)，我们通过 for 循环分别计算每一个先验值的后验：\n  \n  `for prior_θ,prior in zip(prior_θs,priors)`\n\n  因为每一个先验参数有不同的概率，我们通过`zip`函数将两个变量打包在一起，使得可以同时循环两个变量。\n- 为了保存循环计算得到的多个后验值，我们把各值加入到一个空数组`unstd_posteriors = []`\n\n  即下面代码`unstd_posteriors += [unstd_posterior]`"},{"cell_type":"code","execution_count":37,"metadata":{"collapsed":false,"id":"BEE473B7CF5A446FAB145C7C4078AA7A","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[],"source":"# 定义计算后验的函数\ndef calculate_posterior(prior_θ, prior, data):\n    likelihood  = st.bernoulli(prior_θ).pmf(data).prod()\n    unstd_posterior = likelihood * prior\n    return unstd_posterior\n\nunstd_posteriors = []\nfor prior_θ,prior in zip(prior_θs,priors):\n    unstd_posterior = calculate_posterior(prior_θ,prior,data)\n    unstd_posteriors += [unstd_posterior]"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"D7227757EBB1475683D9D30E26C8E5ED","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"Tips: 上面的循环可以写作下面**更简洁**的形式，这得力于python的语法糖。"},{"cell_type":"code","execution_count":38,"metadata":{"collapsed":false,"id":"B501C4839BB2443A85679A0B03031EC3","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[],"source":"unstd_posteriors = [calculate_posterior(prior_θ,prior,data) for prior_θ,prior in zip(prior_θs,priors)] "},{"cell_type":"code","execution_count":39,"metadata":{"collapsed":false,"id":"70F30147135E40D993E00123C9E1606D","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["未标准化的后验： [0.00044789760000000004, 0.0011117830499999999, 0.0004194303999999999]\n"]}],"source":"print(\"未标准化的后验：\",unstd_posteriors)"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"F3DACE04540548C6828296B8249C4D15","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"此时的后验不是有效的概率分布，我们可以通过计算贝叶斯公式中的分母，从而得到出标准化(valid)的后验概率。"},{"cell_type":"code","execution_count":40,"metadata":{"collapsed":false,"id":"85E9B96EDEA84F41A702B93710FC9CD2","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["标准化的后验： [0.22631252 0.5617588  0.21192868]\n"]}],"source":"posteriors = unstd_posteriors/np.sum(unstd_posteriors)\nprint(\"标准化的后验：\",posteriors)"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"E04814FDE5A64035811C128A7309C68C","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"可以看到，标准化的后验概率总和为1。\n\n并且参数为0.7时，后验概率最大。数据增强了我们对于参数等于0.7的信心(之前我们认为参数等于0.7的概率为0.5)。"},{"cell_type":"code","execution_count":41,"metadata":{"collapsed":false,"id":"554BA7D52C3D41078343B275454077A5","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"data":{"text/plain":["Text(0, 0.5, 'posterior probability')"]},"execution_count":41,"metadata":{},"output_type":"execute_result"},{"data":{"text/html":["<img src=\"https://cdn.kesci.com/upload/rt/554BA7D52C3D41078343B275454077A5/rjvmkxu5cc.png\">"],"text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":"plt.plot(prior_θs,posteriors)\nplt.xlabel(\"θ\")\nplt.ylabel(\"posterior probability\")"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"599AE6B4005D470696021E12B4439DF3","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"**先验为正态分布时，计算后验分布**\n\n同样，我们可以根据上节课的知识，为先验设置一个正态分布，然后再计算后验分布。\n\n我们假设先验参数服从均值为0.5，标准差为0.1的正态分布"},{"cell_type":"code","execution_count":42,"metadata":{"collapsed":false,"id":"D931A5BCF4AF4D4F81771923123A56A0","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[],"source":"prior_θs = np.linspace(0, 1, 100) # 用100个数值 遍历先验参数值从0到1\npriors = st.norm.pdf(prior_θs,0.5,0.1)/np.sum(st.norm.pdf(prior_θs,0.5,0.1)) # 先验参数服从均值为0.5，标准差为0.1的正态分布"},{"cell_type":"code","execution_count":43,"metadata":{"collapsed":false,"id":"278460BECF694C058AF6316FFA6C542A","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"data":{"text/plain":["Text(0, 0.5, 'priors probability')"]},"execution_count":43,"metadata":{},"output_type":"execute_result"},{"data":{"text/html":["<img src=\"https://cdn.kesci.com/upload/rt/278460BECF694C058AF6316FFA6C542A/rjvmljkxis.png\">"],"text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":"plt.plot(prior_θs,priors)\nplt.xlabel(\"prior θ\")\nplt.ylabel(\"priors probability\")"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"AADF76F7C78041888CFCC4AA308D4826","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"我们通过上面定义的计算后验的函数`calculate_posterior` 以及刚提到的 for循环计算每一个先验对应的后验值"},{"cell_type":"code","execution_count":45,"metadata":{"collapsed":false,"id":"27BF6228C8014FF18747E8B023B04095","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[],"source":"unstd_posterior = [calculate_posterior(prior_θ,prior,data) for prior_θ,prior in zip(prior_θs,priors)]"},{"cell_type":"code","execution_count":46,"metadata":{"collapsed":false,"id":"F5EBB7BC6B8140599A86C535089E2231","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"data":{"text/plain":["Text(0, 0.5, 'unstandardized posteriors')"]},"execution_count":46,"metadata":{},"output_type":"execute_result"},{"data":{"text/html":["<img src=\"https://cdn.kesci.com/upload/rt/F5EBB7BC6B8140599A86C535089E2231/rjvmm89frb.png\">"],"text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":"plt.plot(prior_θs,unstd_posterior)\nplt.xlabel(\"θ\")\nplt.ylabel(\"unstandardized posteriors\")"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"D5D2B43115E54F17864B61389DA4130A","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"注意，此时y轴表示的不是后验概率。\n\n因此我们需要根据公式 $P(data) =\\sum_{\\theta}^{} p(data,\\theta)=\\sum_{\\theta}^{} p(data|\\theta)p(\\theta)$ 计算后验概率。\n\n对应的代码 \n\n- $P(data) =$ `np.sum(unstd_posterior)`\n- $p(\\theta|data) = \\frac{p(data|\\theta)p(\\theta)}{p(data)}$ 对应 `posterior = unstd_posterior/np.sum(unstd_posterior)`"},{"cell_type":"code","execution_count":47,"metadata":{"collapsed":false,"id":"047DC11A7D92451180397E97D7C026C9","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"data":{"text/plain":["<matplotlib.legend.Legend at 0x7fbfe2ba2150>"]},"execution_count":47,"metadata":{},"output_type":"execute_result"},{"data":{"text/html":["<img src=\"https://cdn.kesci.com/upload/rt/047DC11A7D92451180397E97D7C026C9/rjvmmrvjb7.png\">"],"text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":"posterior = unstd_posterior/np.sum(unstd_posterior)\n\nplt.plot(prior_θs, priors, color = 'grey',ls = '--',label=\"prior\")\nplt.plot(prior_θs, posterior, color = 'red',label=\"posterior\")\nplt.legend()"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"BD043EA8D088434AA28737C4A12CE3F1","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"### MCMC原理实现过程及其代码"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"885CD737B66140BF9C4B2BDBF7F906A9","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"在前面的代码示例中，存在两个问题：\n- **我们遍历了所有先验参数的可能性去计算后验分布，然而当参数不止一个时，遍历所有参数非常消耗计算量。**\n  \n  对应的代码为 `prior_θs = np.linspace(0, 1, 100)`。可以想象，如果参数不止一个，那么prior_θs将不再是一个一维向量。\n\n- **对于复杂的后验分布，公式的分母部分难以计算，所以我们无法得到标准化的后验分布**\n\n  对应代码部分 `np.sum(unstd_posterior)`。可以想象，如果参数不止一个，我们需要对每一个参数都进行求和操作。\n\n\n使用MCMC的**意义**就在于：\n\n通过采样的方法避免遍历所有的参数值，并且**避免计算分母**。"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"BEB1D518A986434883CBB3EB4202122F","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"因此，MCMC的关键在于**如何在后验分布中进行采样**？"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"F72873E208084E8893E61FA4953B5DF9","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"**为了了解采样的概念，我们首先从参数范围中随机进行采样**\n\n我们知道参数(海面覆盖率)的范围为0到1，因此我们可以从0到1中随机抽取参数"},{"cell_type":"code","execution_count":48,"metadata":{"collapsed":false,"id":"9835038BCD464FF29557E98D778C3CC0","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[],"source":"samples = st.uniform(0, 1).rvs(100) # 我们从一个0到1的均匀分布中采样100次，用samples代表采集到的样本"},{"cell_type":"code","execution_count":49,"metadata":{"collapsed":false,"id":"BF6C01734C114A1D8A7D9FFD14AE9CAC","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[],"source":"# 我们也可以通过for循环达到同样的效果\nn_iters = 100\nsamples = {\"θ\":np.zeros(n_iters)}\nfor n_iter in range(n_iters):\n  samples[\"θ\"][n_iter] = st.uniform(0,1).rvs(1) # 此时，每次采样一次"},{"cell_type":"code","execution_count":50,"metadata":{"collapsed":false,"id":"DCD4243127084E80902D8C5F77810BA5","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"data":{"text/plain":["(array([14.,  7., 11.,  9., 13., 10., 14.,  4.,  8., 10.]),\n"," array([0.00567931, 0.10238004, 0.19908078, 0.29578152, 0.39248226,\n","        0.489183  , 0.58588374, 0.68258448, 0.77928522, 0.87598595,\n","        0.97268669]),\n"," <BarContainer object of 10 artists>)"]},"execution_count":50,"metadata":{},"output_type":"execute_result"},{"data":{"text/html":["<img src=\"https://cdn.kesci.com/upload/rt/DCD4243127084E80902D8C5F77810BA5/rjvmob9hp3.png\">"],"text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":"plt.hist(samples[\"θ\"])"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"A2352E5C36F6436791861D435B085CCA","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"我们可以看到，随机采集的参数分布与真实的后验分布差距很大。"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"47A9082FF43E41B9AF52BFBE2D1D8F59","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"### 拒绝采样(reject sampling)"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"3606F86621CF4AAD9A9485443FE76A0A","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"**通过前面学到的*拒绝采样*算法，我们可以优化采集到的样本，使得它更接近后验样本**\n\n具体思路为，我们会利用未标准化的后验，当该后验对应的y轴的值越大时，我们越有可能保留对应的参数。"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"6C60D5801D1E482AAD90A256B524831D","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"这里我们重新定义计算后验的函数。\n\n其中，我们定义先验分布为正态分布。\n并且，避免参数值超过0-1的范围。"},{"cell_type":"code","execution_count":52,"metadata":{"collapsed":false,"id":"D143B9C4612E4A5AAC9EAA5DC6CFBA67","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[],"source":"def calculate_posterior(θ, data):\n    if 0 <= θ <= 1:                           # 避免参数值超过0-1的范围\n      prior = st.norm(0.5, 0.1).pdf(θ)        # 定义先验分布是均值为0.5，标准差是0.1的正态分布。\n      likelihood  = st.bernoulli(θ).pmf(data).prod()\n      unstd_posterior = likelihood * prior\n    else:\n        unstd_posterior = -np.inf\n        \n    return unstd_posterior"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"A8A8FCC38A464B8087721D15A1589A9D","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"- 首先，我们从0到1中随机生成一个参数值：`proposal_θ = st.uniform(0,1).rvs(1)`\n- 然后，我们计算该参数值的非标准后验：\n  \n  `unstd_posterior = calculate_posterior(proposal_θ, prior_θ, data)`。\n  \n  这里面我们的数据为10次抛地球仪，获得7次海面向上；先验服从均值为0.5标准差为0.1的正态分布。\n- 最后，我们生成一个随机数，如果非标准后验的值大于该随机数，我们就接受该参数采样，否则我们保持之前的参数采样：\n  \n  ```\n  if unstd_posterior*1000 > st.uniform(0, 1).rvs(1):\n        θ = proposal_θ\n  else:\n        θ = samples['θ'][n_iter-1]\n  ```\n        \n  可以想象，非标准后验的值越大，对应的参数样本越有可能被保留"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"9224FE7257AF4B3CACFCD6D44550664B","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"我们通过一个**练习**来了解，哪些后验参数容易被接受：\n\n这里我们设置拒绝阈限为0.5 `reject_threshold = 0.5`，\n\n当后验参数对应的y轴数值大于该值，我们接受该后验参数 \n\n```\nif unstd_posterior > reject_threshold: \n  print(\"接受的参数：\",θ)\n```"},{"cell_type":"code","execution_count":53,"metadata":{"collapsed":false,"id":"F091A0C41F13489D9A82AD2F5C5931A3","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"ename":"TypeError","evalue":"'<=' not supported between instances of 'int' and 'ellipsis'","output_type":"error","traceback":["\u001b[0;31m---------------------------------------------------------------------------\u001b[0m","\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)","\u001b[0;32m/tmp/ipykernel_48/1856522107.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     11\u001b[0m \u001b[0;31m# 根据数据与先验参数计算后验\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     12\u001b[0m \u001b[0mdata\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 13\u001b[0;31m \u001b[0munstd_posterior\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcalculate_posterior\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mθ\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     14\u001b[0m \u001b[0munstd_posterior\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0munstd_posterior\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0;36m1000\u001b[0m \u001b[0;31m# 因为unstd_posterior太小，我们将其扩大1000倍\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     15\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/tmp/ipykernel_48/2987578944.py\u001b[0m in \u001b[0;36mcalculate_posterior\u001b[0;34m(θ, data)\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mcalculate_posterior\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mθ\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m     \u001b[0;32mif\u001b[0m \u001b[0;36m0\u001b[0m \u001b[0;34m<=\u001b[0m \u001b[0mθ\u001b[0m \u001b[0;34m<=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m:\u001b[0m                           \u001b[0;31m# 避免参数值超过0-1的范围\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m       \u001b[0mprior\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mst\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnorm\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m0.5\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0.1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpdf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mθ\u001b[0m\u001b[0;34m)\u001b[0m        \u001b[0;31m# 定义先验分布是均值为0.5，标准差是0.1的正态分布。\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m       \u001b[0mlikelihood\u001b[0m  \u001b[0;34m=\u001b[0m \u001b[0mst\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbernoulli\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mθ\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpmf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprod\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m       \u001b[0munstd_posterior\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlikelihood\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0mprior\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;31mTypeError\u001b[0m: '<=' not supported between instances of 'int' and 'ellipsis'"]}],"source":"import numpy as np               # numpy 是专门用于数组运算的包\nimport scipy.stats as st         # 从scipy.stats里载入分布函数\nimport matplotlib.pyplot as plt  # matplotlib.pyplot 是专门用于画图的包\n\n########################################################\n# 练习\n# 请尝试通过 st.uniform(0, 1).rvs(1) 随机生成一个0-1的参数\n########################################################\nθ = ...\n\n# 根据数据与先验参数计算后验\ndata = [1,1,1,1,1,0,0,1,1,0]\nunstd_posterior = calculate_posterior(θ, data)\nunstd_posterior = unstd_posterior*1000 # 因为unstd_posterior太小，我们将其扩大1000倍, (rethinkg: 为什么？)\n\n# 设定拒绝阈限\nreject_threshold = 0.5\n\nprint(\"参数y轴值：\",unstd_posterior,\"拒绝阈限\",reject_threshold)\nif unstd_posterior > reject_threshold:\n    print(\"接受的参数：\",θ)\nelse:\n    print(\"拒绝的参数：\",θ)"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"DE1CF34C55F84400A9655ABCAF8F2723","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"可以发现：该方法与随机抽样**最主要的区别**在于，利用随机抽取的样本计算后验，之后根据后验选择是否接受后验参数"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"726DE99AA9C640008154E40985D865E7","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"我们通过for循环尝试生成1000个随机的参数(proposal_θ):\n```\nn_iters = 1000\nfor n_iter in range(n_iters):\n  proposal_θ = st.uniform(0,1).rvs(1)\n```\n\n如果参数对应y轴数值的1000倍大于0.5我们接受该参数，否则我们拒绝该参数：\n```\nreject_threshold = 0.5\nif unstd_posterior*1000 > reject_threshold:\n  θ = proposal_θ\n  samples['θ'][n_iter] = θ\n```"},{"cell_type":"code","execution_count":54,"metadata":{"collapsed":false,"id":"25B504340DDF4A6D8FED4F05CF5EAF6F","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[],"source":"data = [1,1,1,1,1,1,1,0,0,0] # 数据与之前相同，10次抛地球仪，我们获得7次海面向上\nn_iters = 1000\nsamples = {\"θ\":np.zeros(n_iters)}\n\nfor n_iter in range(n_iters):\n    proposal_θ = st.uniform(0,1).rvs(1)\n    unstd_posterior = calculate_posterior(proposal_θ, data)\n    \n    reject_threshold = 0.5\n    if unstd_posterior*1000 > reject_threshold:\n        θ = proposal_θ\n    else:\n        θ = samples['θ'][n_iter-1]\n\n    samples['θ'][n_iter] = θ"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"A6A96767D7244EF199E6041769AC458C","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"我们可以对该分布进行统计推断，如同上一次课程我们对后验分布进行统计推断一样。"},{"cell_type":"code","execution_count":55,"metadata":{"collapsed":false,"id":"782A8C018FD54A6C8A37EDA29582674A","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"data":{"text/plain":["<AxesSubplot:title={'center':'θ'}>"]},"execution_count":55,"metadata":{},"output_type":"execute_result"},{"data":{"text/html":["<img src=\"https://cdn.kesci.com/upload/rt/782A8C018FD54A6C8A37EDA29582674A/rjvmriam6t.png\">"],"text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":"az.plot_posterior(samples) # 绘制样本的分布"},{"cell_type":"code","execution_count":56,"metadata":{"collapsed":false,"id":"FA73CED8AFCF44278C76AC0A48EF944F","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"name":"stderr","output_type":"stream","text":["arviz - WARNING - Shape validation failed: input_shape: (1, 1000), minimum_shape: (chains=2, draws=4)\n"]},{"data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>mean</th>\n","      <th>sd</th>\n","      <th>hdi_3%</th>\n","      <th>hdi_97%</th>\n","      <th>mcse_mean</th>\n","      <th>mcse_sd</th>\n","      <th>ess_bulk</th>\n","      <th>ess_tail</th>\n","      <th>r_hat</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>θ</th>\n","      <td>0.556</td>\n","      <td>0.111</td>\n","      <td>0.391</td>\n","      <td>0.727</td>\n","      <td>0.007</td>\n","      <td>0.005</td>\n","      <td>226.0</td>\n","      <td>247.0</td>\n","      <td>NaN</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["    mean     sd  hdi_3%  hdi_97%  mcse_mean  mcse_sd  ess_bulk  ess_tail  \\\n","θ  0.556  0.111   0.391    0.727      0.007    0.005     226.0     247.0   \n","\n","   r_hat  \n","θ    NaN  "]},"execution_count":56,"metadata":{},"output_type":"execute_result"}],"source":"az.summary(samples) # 计算样本的均值与HDI"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"8700B7B8CD874962A4B494C66CE5EC51","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"从结果看，采集的参数大多位于0.5-0.7, 这符合真实后验分布的结果。\n\n但该分布并不平滑，并且不太符合正态分布，说明我们的算法需要得到优化。"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"664B4545E1C6487D8DB4A9851F180C2D","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"重要性采样有两个问题：\n1. 我们采样参数的过程是随机的`proposal_θ = st.uniform(0,1).rvs(1)`，这样采样参数在分布中peak位置的数量较少，导致采样的效率低。\n2. 如果我们设置的**拒绝阈限**太大，那么我们采集到的数据大多都会被拒绝。"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"A7A9206ED81B421E9A892F1DD8864AE0","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"通过**练习**了解不同**拒绝阈限**对接受的样本数量的影响"},{"cell_type":"code","execution_count":57,"metadata":{"collapsed":false,"id":"9972CF941EEF4CD7878C29C57757E4FC","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"ename":"TypeError","evalue":"'>' not supported between instances of 'float' and 'ellipsis'","output_type":"error","traceback":["\u001b[0;31m---------------------------------------------------------------------------\u001b[0m","\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)","\u001b[0;32m/tmp/ipykernel_48/2614330120.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     13\u001b[0m     \u001b[0munstd_posterior\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcalculate_posterior\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mproposal_θ\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     14\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 15\u001b[0;31m     \u001b[0;32mif\u001b[0m \u001b[0munstd_posterior\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0;36m1000\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0mreject_threshold\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     16\u001b[0m         \u001b[0msamples_accept\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mproposal_θ\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     17\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;31mTypeError\u001b[0m: '>' not supported between instances of 'float' and 'ellipsis'"]}],"source":"n_iters = 1000\nsamples_accept = []\nsamples_reject = []\n\n########################################################\n# 练习\n# 请尝试设置 reject_threshold 为0-1的任何值\n########################################################\nreject_threshold = ...\n\nfor n_iter in range(n_iters):\n    proposal_θ = st.uniform(0,1).rvs(1)\n    unstd_posterior = calculate_posterior(proposal_θ, data)\n    \n    if unstd_posterior*1000 > reject_threshold:\n        samples_accept += [proposal_θ[0]]\n    else:\n        samples_reject += [proposal_θ[0]]\n\naccept_number = [len(samples_accept), len(samples_reject)]\nprint(\"接受的样本数量：\", accept_number[0], \"拒绝的样本数量：\", accept_number[1])\n\nfig,ax = plt.subplots()\nax.bar([\"accept\",\"reject\"],accept_number)\nplt.show()"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"3C140969933B490198DBC4D18BE9AD79","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"**Metropolis-Hastings (HM) 算法**\n\n为了提高采样的效率，减少被拒绝的采样，我们可以使用基于 MCMC 思想的 Metropolis-Hastings (HM) 算法。\n\n其与接受拒绝采样的**区别在于**，我们接受采样的概率与之前的采样的后验有关。\n\n`ratio = unstd_posterior1 / unstd_posterior0`  如果当前采样`unstd_posterior1`的后验的概率(未标准化)大于上一次采样的后验`unstd_posterior0`，那么两次后验的比值 ratio 值越大，那么越有可能接受当前采样的参数。\n\n并且，我们每次采样不是随机采样，而是在当前后验参数附近采样 (根据马尔科夫的性质)：\n\n`proposal_θ1 = st.norm(proposal_θ0, 0.5).rvs(1)` 当前参数 `proposal_θ1` 是从之前参数 `proposal_θ0` 附近采样。\n\n这种根据先前采样的建议(proposal)得到的采样，称为 proposal 参数。 对应采样的分布 `st.norm(proposal_θ0,0.5)` 称为 proposal distribution. \n\n注意，这个分布里的参数可以自己设定，比如，标准差0.5；或者是将标准差也作为模型的参数；并且 proposal distribution的选择也具有人为性。 这些人为性都影响采样的效率。"},{"cell_type":"code","execution_count":58,"metadata":{"collapsed":false,"id":"9721CAC50F684A69ACE85A7D2C53BFEE","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[],"source":"n_iters = 1000\nsamples = {\"θ\":np.zeros(n_iters)}\nproposal_theta1 = 0          # 为当前迭代下的proposal_θ\nproposal_theta0 = 0          # 为上一个迭代的proposal_θ\nunstd_posterior1 = 0.001 # 为当前迭代下的unstd_posterior\nunstd_posterior0 = 0.001 # 为上一个迭代的unstd_posterior\n\nfor n_iter in range(n_iters):\n    prop_theta1 = st.norm(prop_theta0,0.5).rvs(1)\n    unstd_posterior1 = calculate_posterior(prop_theta1, data)\n    \n    ratio = unstd_posterior1 / unstd_posterior0\n\n    if ratio > st.uniform(0, 1).rvs(1):\n        prop_theta0 = prop_theta1\n        unstd_posterior0 = unstd_posterior1\n        \n    samples['θ'][n_iter] = prop_theta0"},{"cell_type":"code","execution_count":59,"metadata":{"collapsed":false,"id":"D374F02A96BC4F39BDC2108908402CF3","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"data":{"text/plain":["<AxesSubplot:title={'center':'θ'}>"]},"execution_count":59,"metadata":{},"output_type":"execute_result"},{"data":{"text/html":["<img src=\"https://cdn.kesci.com/upload/rt/D374F02A96BC4F39BDC2108908402CF3/rjvmtjaleh.png\">"],"text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":"az.plot_posterior(samples) # 绘制样本的分布"},{"cell_type":"code","execution_count":60,"metadata":{"collapsed":false,"id":"9AF6D5894DF54196BC3483E530F70CC4","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"name":"stderr","output_type":"stream","text":["arviz - WARNING - Shape validation failed: input_shape: (1, 1000), minimum_shape: (chains=2, draws=4)\n"]},{"data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>mean</th>\n","      <th>sd</th>\n","      <th>hdi_3%</th>\n","      <th>hdi_97%</th>\n","      <th>mcse_mean</th>\n","      <th>mcse_sd</th>\n","      <th>ess_bulk</th>\n","      <th>ess_tail</th>\n","      <th>r_hat</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>θ</th>\n","      <td>0.559</td>\n","      <td>0.085</td>\n","      <td>0.39</td>\n","      <td>0.711</td>\n","      <td>0.007</td>\n","      <td>0.005</td>\n","      <td>147.0</td>\n","      <td>117.0</td>\n","      <td>NaN</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["    mean     sd  hdi_3%  hdi_97%  mcse_mean  mcse_sd  ess_bulk  ess_tail  \\\n","θ  0.559  0.085    0.39    0.711      0.007    0.005     147.0     117.0   \n","\n","   r_hat  \n","θ    NaN  "]},"execution_count":60,"metadata":{},"output_type":"execute_result"}],"source":"az.summary(samples) # 计算样本的均值与HDI"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"71FA9C2CE6284C318CA7E45080946B0D","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"可以看到，我们顺利得到了参数的后验分布，并且该结果比重要性采样算法的结果更好。\n\n有了该采样样本，我们就可以用上一节课学习到的统计推断方法分析该采样分布了"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"8E0A2626AFB44BF985CE117570C3747F","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"同样，我们可以比较 proposal参数在metroplis算法中的接受频率。"},{"cell_type":"code","execution_count":62,"metadata":{"collapsed":false,"id":"A1EF8D60879E40AA86B6C8E7DDA023B4","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["接受的样本数量： 708 拒绝的样本数量： 292\n"]},{"data":{"text/html":["<img src=\"https://cdn.kesci.com/upload/rt/A1EF8D60879E40AA86B6C8E7DDA023B4/rjvmumke3v.png\">"],"text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":"n_iters = 1000\nsamples_accept = []\nsamples_reject = []\n\nfor n_iter in range(n_iters):\n    proposal_θ1 = st.norm(proposal_θ0,0.5).rvs(1)\n    unstd_posterior1 = calculate_posterior(proposal_θ1, data)\n    \n    ratio = unstd_posterior1 / unstd_posterior0\n\n    if ratio < st.uniform(0, 1).rvs(1):\n        samples_accept += [proposal_θ[0]]\n    else:\n        samples_reject += [proposal_θ[0]]\n\naccept_number = [len(samples_accept), len(samples_reject)]\nprint(\"接受的样本数量：\", accept_number[0], \"拒绝的样本数量：\", accept_number[1])\n\nfig,ax = plt.subplots()\nax.bar([\"accept\",\"reject\"],accept_number)\nplt.show()"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"86DE9D5536FD4B159F4AAA338719745D","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"可以看到，metroplis算法的接受效率更好。"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"363217A6F9AF444B85093364D5742448","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"需要注意的是，通过MCMC采样得到的是从后验分布中采样得到的参数样本，而不是关于后验分布的函数。\n\n后验分布参数样本相比于后验分布函数的优点在于：\n- 方便计算后验分布的平均值等统计值\n\n缺点在于：\n- 如果样本无法代表后验分布，基于后验分布的统计推断也会出错"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"3B7013EEA44A4CBAA69E461440070579","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"### pymc3\n\n显然通过写for循环来实现MCMC算法还是过于复杂，我们下节课将结合 pymc工具包，简化以上计算。"},{"cell_type":"code","execution_count":63,"metadata":{"collapsed":false,"id":"8DE4EE2C8CB24FE0827CC25243DDF3C5","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"name":"stderr","output_type":"stream","text":["WARNING (theano.link.c.cmodule): install mkl with `conda install mkl-service`: No module named 'mkl'\n","Auto-assigning NUTS sampler...\n","Initializing NUTS using jitter+adapt_diag...\n","Multiprocess sampling (4 chains in 4 jobs)\n","NUTS: [θ]\n"]},{"data":{"text/html":["\n","<style>\n","    /* Turns off some styling */\n","    progress {\n","        /* gets rid of default border in Firefox and Opera. */\n","        border: none;\n","        /* Needs to be in here for Safari polyfill so background images work as expected. */\n","        background-size: auto;\n","    }\n","    progress:not([value]), progress:not([value])::-webkit-progress-bar {\n","        background: repeating-linear-gradient(45deg, #7e7e7e, #7e7e7e 10px, #5c5c5c 10px, #5c5c5c 20px);\n","    }\n","    .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n","        background: #F44336;\n","    }\n","</style>\n"],"text/plain":["<IPython.core.display.HTML object>"]},"metadata":{},"output_type":"display_data"},{"name":"stderr","output_type":"stream","text":["Sampling 4 chains for 1_000 tune and 1_000 draw iterations (4_000 + 4_000 draws total) took 4 seconds.\n"]}],"source":"import pymc3 as pm\n\ndata = [1,1,1,1,1,0,0,1,1,0]\nwith pm.Model() as model:\n    # 设定先验\n    θ = pm.Normal(\"θ\", 0.5, 0.1)\n    # 设定似然函数与数据\n    pm.Binomial(\"samples\", n=1, p=θ, observed=data)\n    # Sample from the posterior distribution\n    samples = pm.sample(1000, return_inferencedata=True)"},{"cell_type":"code","execution_count":64,"metadata":{"collapsed":false,"id":"C2510125C483472D9BEDAB223C372680","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"data":{"text/plain":["<AxesSubplot:>"]},"execution_count":64,"metadata":{},"output_type":"execute_result"},{"data":{"text/html":["<img src=\"https://cdn.kesci.com/upload/rt/C2510125C483472D9BEDAB223C372680/rjvmxc8mfo.png\">"],"text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":"az.plot_dist(samples.posterior[\"θ\"])"},{"cell_type":"code","execution_count":65,"metadata":{"collapsed":false,"id":"4C6122809DA246538A27F372FBC24F61","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"outputs":[{"data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>mean</th>\n","      <th>sd</th>\n","      <th>hdi_3%</th>\n","      <th>hdi_97%</th>\n","      <th>mcse_mean</th>\n","      <th>mcse_sd</th>\n","      <th>ess_bulk</th>\n","      <th>ess_tail</th>\n","      <th>r_hat</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>θ</th>\n","      <td>0.559</td>\n","      <td>0.086</td>\n","      <td>0.393</td>\n","      <td>0.72</td>\n","      <td>0.002</td>\n","      <td>0.001</td>\n","      <td>1900.0</td>\n","      <td>2332.0</td>\n","      <td>1.0</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["    mean     sd  hdi_3%  hdi_97%  mcse_mean  mcse_sd  ess_bulk  ess_tail  \\\n","θ  0.559  0.086   0.393     0.72      0.002    0.001    1900.0    2332.0   \n","\n","   r_hat  \n","θ    1.0  "]},"execution_count":65,"metadata":{},"output_type":"execute_result"}],"source":"az.summary(samples)"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"B2ED33701C7F4469ABBC37660E732B0E","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"从PyMC3的采样结果来看，同样采集1000个样本，PyMC3的后验分布明显更加平滑和准确。\n\n这提示我们，即使我们知道采样器的算法和原理，但我们自己写的代码仍然和专业工具包存在差异。\n\n因此更重要的是，我们的学习目标是如何利用优秀的算法和工具完成数据分析，而不是自己研究如何创造算法和工具"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"773E0003D0444024A691E77A35D0D27E","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"总结，MCMC算法的专业术语，包括：\n\n- MCMC samplers：由于MCMC算法的本质是对后验分布进行采样，因此该方法也被叫做MCMC采样器(samplers)。\n- iteration，draws，samples：MCMC采样器在采样过程中需要进行循环，每个循环称为一个iteration，每个iteration中采集的参数称作样本samples，也可以叫做draws，就像从抽屉里抽出来一样。\n- proposal dsitribution：即参数采样的过程不再是随机采样，而是根据建议分布(proposal dsitribution)进行采样。\n- chain：在使用MCMC算法时，我们会同时运行多个“循环”，每一个循环为一条马尔科夫链(chain)。运行多条链的目的在于，可以更高效的采集更多的样本，也可以避免单条链采到的样本不能代表后验分布的影响。\n"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"A9C83713122B4FF199CA171CBFB8C8FD","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"### HMC (Hamiltonian Monte Carlo)"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"06C377E7754B40F7A79E32234C7CA026","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"Metropolis-Hastings (HM)的优点：\n- MH算法步骤非常精简，每次迭代iteration只有两个步骤\n   1. 根据上次采样的参数生成proposal distribution，然后进行采样\n\n   `proposal_θ1 = st.norm(proposal_θ0,0.5).rvs(1)`\n\n   2. 计算拒绝概率(接受／拒绝)\n\n   ```\n   unstd_posterior1 = calculate_posterior(proposal_θ1, data)\n   ratio = unstd_posterior1 / unstd_posterior0\n   if ratio < st.uniform(0, 1).rvs(1): print(\"接受\")\n   ```\n\n- MH算法可以从复杂的后验分布中采样，虽然其效率不高。"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"ADBA69C4AC4E42D580DF47FCBD129351","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"Metropolis-Hastings (HM)的问题：\n- proposal distribution 的标准差太大会导致大量被拒绝的样本，而太小的标准差会导致花费更长时间来“探索分布”\n\n  `proposal_θ1 = st.norm(proposal_θ0,0.5).rvs(1)` 比如将这里的0.5修改为5\n\n- 当模型参数很多的时候，MH 采样的效率非常低，因为它的采样是根据 proposal distribution随机选取的\n- 对于高维后验分布，MH 的采样是非常不均匀的"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"2613B52F2DB142C1B8F14C04DB16DB86","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"\n![Image Name](https://cdn.kesci.com/upload/image/rjvgyvcsre.gif?imageView2/0/w/640/h/640)\n\n\n绿色为接受的采样，红色是拒绝的采样。可以看到HM算法中被拒绝的采样是比较多的。"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"530EF135E2714375881A36CD1B0356B2","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"**HMC (Hamiltonian Monte Carlo)**\n\n为了解决 HM 中 proposal distribution 由于随机采样导致效率低的问题，研究者提出了HMC。"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"669513842E2042269DB68C6969B59C98","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"HMC结合物理概念来优化 MCMC：\n- HMC 通过基本物理规律 **能量=势能+动能** 假设采样的分布(后验分布)为系统的总能量，然后将该系统分为两个子部分\n- 由于动量增加，势能就会减少，因此**系统总能量不会变化**。这说明我们采样的分布不会因为系统的动态变化而变化，这类似于马尔科夫的平稳分布\n\n\n![Image Name](https://cdn.kesci.com/upload/image/rjviiji42v.gif?imageView2/0/w/640/h/640)\n"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"71E811206F954DDA828FEF9E3584FF06","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"- 势能，与位置有关，比如对于重力势能，高度越高，势能越大。相似的，可以把参数的位置看作高度，把参数对应的似然看作势能。可以结合拒绝采样进行理解，在拒绝采样中，似然值越大对应的参数越有可能被接受，因此，这里势能越大的参数也越有可能被接受。\n\n\t```\n\tunstd_posterior0 = calculate_likelihood(proposal_θ0, data)  # 上一次参数对应的似然值(即势能)\n\tunstd_posterior1 = calculate_likelihood(proposal_θ1, data)  # 当前次参数对应的似然值\n\tratio = unstd_posterior1 / unstd_posterior0                          # 与 HM一样计算两次参数似然的比值\n\t``` \n\t\n- 动能，与速度有关，比如动能公式 $\\frac{1}{2}mv^2$。其对应的是似然函数的梯度，从下图中可以发现，沿着梯度进行采样可以更多的采样到分布 peak 位置部分的参数。\n\t\n\t```\n\tmomentum0 = momentum_dist.rvs()                                                 # 随机生成动能值\n\tgradient = calculate_gradient(momentum0, likelihood_function)     # 根据动能值计算似然函数的梯度\n\tmomentum1 = momentum0 + gradient                                            # 根据梯度更新动能\n\t\n\t# 根据\n\tmomentum_ratio = calculate_likelihood(momentum0, data)/(momentum1, data)\n\t```\n"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"4DD1A9140DD644578CEA857F62EC3785","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"- 需要注意的是，势能会随着动能的变化而变化：\n\n```\nproposal_θ1 = unstd_posterior0 + momentum                                # 根据动能更新参数位置\nunstd_posterior1 = calculate_likelihood(proposal_θ1, data)             # 根据更新的位置计算势能\n```\n\n- 最后结合两部分能量进行判断\n\n```\nif ratio*momentum_ratio < st.uniform(0, 1).rvs(1): print(\"接受\")\n```"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"8F06BF76073D4D4789F7BBB767186B71","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"\n![Image Name](https://cdn.kesci.com/upload/image/rjvh3zx4an.gif?imageView2/0/w/640/h/640)\n\n\n绿色为接受的采样，红色是拒绝的采样。可以看到HMC几乎没有被拒绝的采样，其采样效率高。\n\n红色的线条是 HMC 表示沿着与梯度有关的 proposal distritbution 进行采样。"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"0B3353F2CCC546F78BD37255FBCFE61D","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"HMC 与 HM 相比的**主要区别**在于：\n- HMC 凭借更少的迭代数来获得相对多的具有代表性的采样\n- HMC 因为每次单次迭代需要计算似然函数的梯度，因此计算的“消耗”更高，但 HMC 仍然比HM效率更高\n- HMC 在大多数情况下接受采样的概率比 HM 高"},{"cell_type":"markdown","metadata":{"collapsed":false,"id":"A25EAB293BED46E986FC3BE9F2F7F17E","jupyter":{},"notebookId":"634d0eaaa57b37b7a0bfc55b","scrolled":false,"slideshow":{"slide_type":"slide"},"tags":[],"trusted":true},"source":"基于HMC，最常用的算法是 No-U-Turn sampler（NUTS），pymc 和 Stan默认使用的也是NUTS算法。\n\n\n![Image Name](https://cdn.kesci.com/upload/image/rjvix0jc6b.png?imageView2/0/w/320/h/320)\n\n\n![Image Name](https://cdn.kesci.com/upload/image/rjviyt1d5d.png?imageView2/0/w/320/h/320)"},{"cell_type":"markdown","metadata":{"id":"8A7B7F1A34BE4F09ABD848C15A298A86","jupyter":{},"collapsed":false,"scrolled":false,"tags":[],"slideshow":{"slide_type":"slide"},"notebookId":"634d0eaaa57b37b7a0bfc55b","trusted":true},"source":"**吉布斯采样 (Gibbs sampling)**\n\nGibbs sampling 通过从参数的条件分布进行采样或从给定的另一个参数值的概率分布进行采样。"},{"cell_type":"markdown","metadata":{"id":"6516E6CD54504CE299AFAB882613F949","jupyter":{},"collapsed":false,"scrolled":false,"tags":[],"slideshow":{"slide_type":"slide"},"notebookId":"634d0eaaa57b37b7a0bfc55b","trusted":true},"source":"Gibbs sampling 的目的是从难以直接采样的多元概率分布P(x)中生成样本。这个算法基于每一个条件分布$p(x_i|x_i)$进行采样"},{"cell_type":"markdown","metadata":{"id":"3AD015D34E2D423F8F839924EA98431D","jupyter":{},"collapsed":false,"scrolled":false,"tags":[],"slideshow":{"slide_type":"slide"},"notebookId":"634d0eaaa57b37b7a0bfc55b","trusted":true},"source":"使用Gibbs sampling对二元高斯分布进行采样\n\n![Image Name](http://gorayni.github.io/assets/posts/gibbs/gibbs2.gif)\n"},{"cell_type":"markdown","metadata":{"id":"0D0E308A37FB42DB941CF8AB9905E76B","jupyter":{},"collapsed":false,"scrolled":false,"tags":[],"slideshow":{"slide_type":"slide"},"notebookId":"634d0eaaa57b37b7a0bfc55b","trusted":true},"source":"相比于MH算法，Gibbs采样不需要考虑多参数的情况，它将接受拒绝采样分别用于每个步骤。"},{"cell_type":"markdown","metadata":{"id":"C26215696E5D4168936728D7E8047B18","jupyter":{},"collapsed":false,"scrolled":false,"tags":[],"slideshow":{"slide_type":"slide"},"notebookId":"634d0eaaa57b37b7a0bfc55b","trusted":true},"source":"**差分进化算法 (Differential evolution)**\n\n尽管Gibbs采样可以从条件分布进行采样，来获得函数间的相关，但这种采样方法很慢。\n\n还有一个问题就是不相关的建议分布可能与目标分布不匹配。这种不匹配意味着采样时拒绝率会很高。\n\n差分进化算法使用多个链进行采样，并利用各链之间的非独立性来解决参数的相关问题。\n\n"},{"cell_type":"markdown","metadata":{"id":"CCA817FA46364D8596406219971B2CB4","jupyter":{},"collapsed":false,"scrolled":false,"tags":[],"slideshow":{"slide_type":"slide"},"notebookId":"634d0eaaa57b37b7a0bfc55b","trusted":true},"source":"差分进化算法使用其他链之间样本点的距离来对当前链的值进行更新。"},{"cell_type":"markdown","metadata":{"id":"130FDE2EFCCA435D9DE5650B6EB4BA3A","jupyter":{},"collapsed":false,"scrolled":false,"tags":[],"slideshow":{"slide_type":"slide"},"notebookId":"634d0eaaa57b37b7a0bfc55b","trusted":true},"source":"\n![Image Name](https://matteding.github.io/images/diff_evol.gif)\n"},{"cell_type":"markdown","metadata":{"id":"DF6005D4F889470AAEEBF004C0DF8FF5","jupyter":{},"collapsed":false,"scrolled":false,"tags":[],"slideshow":{"slide_type":"slide"},"notebookId":"634d0eaaa57b37b7a0bfc55b","trusted":true},"source":"其中，$\\lambda$为预设值，$\\lambda$的取值会影响采样的速度。"},{"cell_type":"markdown","metadata":{"id":"E1513164AF6D4972A5E0CF9C413F9FCA","jupyter":{},"collapsed":false,"scrolled":false,"tags":[],"slideshow":{"slide_type":"slide"},"notebookId":"634d0eaaa57b37b7a0bfc55b","trusted":true},"source":"### 总结\n\n- the idea of approximation\n- MCMC原理\n- 其他采样算法简介"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python","nbconvert_exporter":"python","file_extension":".py","version":"3.5.2","pygments_lexer":"ipython3"}},"nbformat":4,"nbformat_minor":0}